﻿

   Table of Contents
Title Page
Introduction
About This Book
Overview
SBM OR EBM
Science-based medicine throughout time | D Gorski
The Forerunners of EBM | H Hall
The Need for Science-Based Medicine | S Novella
In which I steal a title: Is medicine a scientific enterprise? | D Gorski
Yes, Jacqueline: EBM ought to be Synonymous with SBM. Kimball Atwood.
Science-based medicine and improving patient safety and quality of care. David Gorski.
HHS is weaponizing evidence-based medicine to falsely portray vaccines as unsafe | D Gorski
Of SBM and EBM Redux. Part I: Does EBM Undervalue Basic Science and Overvalue RCTs?. Kimball Atwood
Of SBM and EBM Redux. Part II: Is it a Good Idea to test Highly Implausible Health Claims? Kimball Atwood
Of SBM and EBM Redux. Part III: Parapsychology is the Role Model for “CAM” Research. Kimball Atwood.
Of SBM and EBM Redux. Part IV: More Cochrane and a little Bayes. Kimball Atwood
Of SBM and EBM Redux. Part IV, Continued: More Cochrane and a little Bayes. Kimball Atwood.
The hijacking of evidence-based medicine | D Gorski
Who Cares About Dr. Adam Cifu’s Plan to Fix EBM in Theory, When in Reality, He Elevated the MAHA Doctors Wrecking it Today? | J Howard
An antivaxxer attacks the evidence-based medicine paradigm: Echoes of RFK Jr.’s “gold standard science” | D Gorski
Clinical trials of homeopathy versus “respect for science” | D Gorski
Answering Our Critics, Part 1 of 2 | H Hall
Answering Our Critics, Part 2 of 2: What’s the Harm? | H Hall
Answering Our Critics – Again! | H Hall
Blog Discussion with an SBM Critic. Steven Novella.
Responding to SBM Critics | S Novella
Answering a criticism of science-based medicine. David Gorski.
Answering another criticism of science-based medicine. David Gorski
Do clinical trials work? It depends on what you mean by “work” | D Gorski
To debate or not to debate: The strange bedfellows of Andrew Weil | D Gorski
Lies, damned lies, and…science-based medicine? David Gorski.
Should Evidence-Based Medicine Be Replaced by Interpersonal Medicine? | H Hall
Science-based medicine isn’t just for CAM. The case of ivermectin shows that it never was. | D Gorski
CAM and Evidenced-Based Medicine. Steven Novella.
Has science-based medicine already lost to pseudoscience? | D Gorski
2020 – The Good and the Bad | S Novella
After 15 years of SBM: Lessons learned and what the future holds | D Gorski
2023: The year that the evidence-based medicine (EBM) paradigm was weaponized against vaccines and public health | D. Gorski
SBM in 2023 | S. Novella
So Long 2025 | S Novella
Evidence-based medicine vs. basic science in medical school | D Gorski
“How do you feel about Evidence-Based Medicine?” | H Hall
An SBM Advocate Goes To Washington | S Novella
Is There a Replication Crisis? | S Novella
The Null hypothesis: Gary Null attacks science-based medicine | D Gorski
The Null hypothesis: Gary Null attacks science-based medicine | D Gorski
Gary Null’s Attack on SBM | S Novella
Joe Biden defeated Donald Trump. Now what for science-based medicine? | D Gorski
Acupuncture and Evidence Based Medicine | S Novella
Is There a Role for the Art of Medicine in Science-Based Practice? | C Jones
On humility, confidence, and science-based surgery | D Gorski
Two Viewpoints | M Crislip
“Moneyball,” the 2012 election, and science- and evidence-based medicine | D Gorski
NCCIH surveys physicians on their recommendation of “complementary health approaches,” with depressing results | D Gorski
WHO And The Death of Evidence-Based Medicine | S Novella
Science-Based Medicine Reviews: Science Based Medicine or Evidence Based Medicine

A Critical Guide for the Health Practitioner and Consumer 
2025 Edition

Executive Editor Steven Novella MD
Managing Editor David Gorski MD PhD
Series Editor Mark Crislip MD

   Introduction 
Steven Novella
The world of health care is becoming an increasingly complex and challenging place. The internet is changing the way we generate and access information, and health information is one of the most popular uses of the internet. It represents an unprecedented democratization of information and opinion. 
While there are many advantages and benefits to such information access, there are also many challenges to health professionals, regulators, academics, and consumers. There is a proliferation of health related products and services, with many provocative health claims. Regulations are generally not keeping up, and often have been watered down, in the face of this onslaught of claims and products. 
The consumer is left with an overwhelming array of health claims, many of which challenge and even directly oppose the traditional experts and institutions. 
The mission of Science-Based Medicine (SBM) is to be a source of reliable information and analysis to help navigate the wild west of health information on the internet and elsewhere. The authors who contribute to this series of e-books are all physicians and other health care professionals who have a great deal of experience educating the public about complex medical issues. We especially take on controversial issues and try to expose pseudoscience and fraud wherever possible.
The Philosophy of Science-Based Medicine
The philosophy of SBM, at its core, is simple: Safe and effective health care is critical to everyone’s quality of life; so much so that it is generally considered a basic human right. Therefore, health care products and services that work and are safe are better than those that do not work or are unsafe. 
The best method for determining which interventions and health products are safe and effective is, without question, good science. Therefore it is in everyone’s best interest for health care to be systematically evaluated by the best science available.
Too often the nature of science itself is misunderstood or misrepresented to the public. Science is not an arcane and privileged discipline. By its very nature it is meant to be transparent and public. Science is nothing more than a systematic and careful use of evidence and logic to evaluate factual claims. And good science possesses certain virtues that are not unique to science but generic to all intellectual endeavors: fairly accounting for all available evidence, using valid and internally consistent logic, using unambiguous concepts and language, proper use of statistics, being quantitatively precise and accurate, and above all being honest.
And yet there are numerous and powerful influences in society that strongly appose the scientific basis of medicine. Driven by some combination of ideology or the desire for profit they wish to eliminate standards of science in health care, or (often under the guise of “health care freedom”) create a double standard in which unscientific methods and products can thrive unchecked. Others simply lack the training or knowledge to achieve minimal standards of quality for scientific medicine. And even the best traditions of scientific medicine can benefit from more critical analysis.
Recognition of the need for careful scientific analysis in medicine partly stems from a recognition that it is very difficult (almost impossible) to reliably assess treatments anecdotally. Human perception, memory, and cognition are prone to a host of flaws, weaknesses, and biases that will tend to mislead us, especially towards conclusions that we wish to be true. Placebo effects, confirmation bias, and simple statistics can make any intervention seem effective, even when they are worthless or harmful. 
Science-Based Medicine vs Evidence-Based Medicine
Within the practice of medicine there is already a recognition of the need to raise the standards of evidence and the availability of the best evidence to the practitioner and the consumer – formalized in the movement known as evidence-based medicine (EBM). 
EBM is a vital and positive influence on the practice of medicine, but it has its limitations. EBM tends to focus on clinical trial results to the exclusion of scientific plausibility. The focus on trial results (which, in the EBM lexicon, is what is meant by “evidence”) has its utility, but fails to properly deal with medical modalities that are highly implausible, or seem to defy well-established findings in science.
All of science describes the same reality, and therefore it must (if it is functioning properly) all be mutually compatible. Collectively, science builds one cumulative model of the natural world. This means we can make rational judgments about what is likely to be true based upon what is already well established. This does not necessarily equate to rejecting new ideas out-of-hand, but rather to adjusting the threshold of evidence required to establish a new claim based upon the prior scientific plausibility of the new claim. Failure to do so leads to conclusions and recommendations that are not reliable, and therefore medical practices that are probably not safe and effective.
The SBM Series
This is why the authors of this series strongly advocate for science based medicine – the use of the best scientific evidence available, in the light of our cumulative scientific knowledge from all relevant disciplines, in evaluating health claims, practices, and products. 
Some of the articles here will analyze individual studies, and explore the strengths and weaknesses of their methods, separating what the study really tells us from how it may be represented in the media. Other articles will summarize what many studies tell us about a specific medical question. 
We will also explore health information on the internet, separating out which information is reliable from that which is misleading, erroneous, or possibly even fraudulent. Likewise we will critique mainstream media coverage of health topics, giving the reader specific tools for to critically analyze health information for themselves.
Other articles will explore topics related to health regulation and academia. Regulations have struggled to keep up with the rapidly changing health care market place, with critical gaps that leave the health care consumer responsible for their own protection. Academic institutions themselves, long the defenders of high standards of scholarship in all fields, including medicine, have been infiltrated by promoters of pseudoscientific belief systems. The result is that there is no longer any single reliable guarantee of scientific integrity. 
The SBM series of e-books is organized by topic, such as acupuncture, chiropractic, homeopathy, and many others. The books are collections of essays, not intended to be a systematic reference, but rather to explore the various scientific issues related to each topic. The goal is to give the reader critical thinking skills and a deeper understanding of, not just the current findings of science, but the process of science, specifically how it relates to health care and the topic of each book. 
Science-based medicine can serve as your pocket guide through the wilderness of health information.

   About This Book

As of 2025, there are over 5000 essays on the Science Based Medicine website. I do no know about you, but I find it difficult to find any post I wrote, much less one written by another contributor. 
So I have updated the books and include almost all the articles through 2025. The volumes are
CAM Concepts
Chiropractic
COVID 
Critical Thinking
Diet and Nutrition
Ethics and CAM
Herbs, Supplements, and Diet
Homeopathy
Infections and Immunity
Law and CAM
Media and Internet
Medicine, Public Health, Dentistry and Pediatrics
Miscellaneous CAM
Naturopathy
Neurology and Psychiatry
Placebo
Quackademic Medicine
Regulation. FDA and FTC
SBM or EBM
Stem Cells, Genetics, AI and Gender
Supplements, Vitamins and Herbs
Vaccine & Anivax
Vaccines and Autism
Vaccines and Preventable Infections
The articles are roughly organized by publication date and topic. Many articles will be in multiple volumes as an essay of, for example, chiropractic and law will be found in both the chiropractic volume and the law volume.
You may find the organization a bit odd. I do. But 4700 plus articles are difficult to organize and I have more to do in my retirement than work of these volumes.
Still, if you find I have missed a nentry or some glaring error, drop me a note and I will fix it for the next edition.
About the authors
Steven P. Novella, MD – Founder and Executive Editor Science Based Medicine.
Founder and currently Executive Editor of Science-Based Medicine Steven Novella, MD.
David H. Gorski, MD, PhD – Managing Editor. Science Based Medicine.
Mark A. Crislip, MD. Editor, Science Based Medicine Guides. 
He did he editing of all the volumes so is to blame for any mistakes.
As to all the other contributors, if you are interested click on the ‘Link’ at top of each essay to go the SBM site, where further information about each author can be found.
A Note On References

The chapters in this book were originally blog entries at ScienceBasedMedicine.org.
Blog and web pages lend themselves to hyperlinks to sources and references that are tedious to translate into footnotes for this ebook.
Under the title of each essay is a hyperlink, cleverly titled ‘Link’, to the original blog entry with all the hyperlinks and references, as well as the comments to the blogs, which often have interesting and thoughtful insights by our readers, as well as some curious comments by the not so thoughtful.
Graphs, tables and links to videos are not in included these essays due to issues of copyright and permission. However, thanks to the ‘Link’, the graphics are a click away.
Thanks for reading the site.
Mark Crislip 


   Overview
There is what to think and there is how to think.
Science and medicine are constantly changing, being refined and evolving as knowedge advances. 
The essays in this collection concern how to think about issues in science based medicine, from the reality behind the placebo effect to how to read the medical literature. It is a primer on how to think and how to avoid thinking badly.

   Critical Thinking: SBM or EBM

Science-based medicine throughout time
David Gorski on December 30, 2013
Link
As 2013 comes to a close, because this probably will be my last post of 2013 (unless, of course, something comes up that I can’t resist blogging about before my next turn a week from now), I had thought of doing one of those cheesy end-of-year lists related to the topic of science-based medicine. Unfortunately, I couldn’t come up with anything I haven’t already done. I even thought of coming up with a list of New Year’s resolutions for 2014. In fact, I even thought of making the first one—in a self-deprecating manner, of course—to be to stop being so mean, nasty, and dogmatic, the better to satisfy my detractors. But then I remembered that nothing is likely to satisfy my detractors and, besides, my ever-lovin’ cuddliness is what makes me so popular. Besides, I have to be me and gotta be true to myself, and all that rot, so that idea went out the window. Of course, what was worse than my inability to come up with something was that I couldn’t think of a way to make it funny. When you’re trying to be funny following the inimitable Mark Crislip, you’d damned well better be funny. So, until my humorous instincts come back, serious it has to be.
But serious doesn’t necessarily mean heavy. The end of a year is a time both to look back on the year before and look forward to the year to come. This year in many ways was a good year for us here at SBM. We launched a Facebook page, reinvigorated our Twitter feed, and have experienced a significant growth in our traffic. Those who know me and/or follow me on various social media know that I’m a big Doctor Who fan, I have been since the 1980s. So the last two big events of the year, the 50th anniversary special in November and the Christmas special on, well, Christmas got me to thinking about time travel, and thinking about time travel revived memories of a topic I covered on my not-so-super-secret other blog four years ago and had been meaning to treat here sometime. It’s a fun topic to finish out the year, not to mention a way for me to blatantly sneak Doctor Who references into an SBM post.
Being a Doctor Who fan and all, not surprisingly, I’ve often wondered what it would be like to be able to travel through time and visit times and places in history that I’m most interested in. For instance, being a World War II buff, I’d certainly want to be able to check out what everyday life was like here in the U.S. during World War II. Given my affinity for psychedelic music and that I was only four years old during most of the Summer of Love, I’d think it cool to check out Haight-Ashbury, although I suspect my reaction to the reality of it would be similar to that of George Harrison when he checked it out for the first time. I guess, if pushed, I’d have to admit that if I were old enough to have been a high school or college student in 1967, I probably would have been one of those straight-laced, short-haired types destined either to go to college to become a doctor or engineer, or to go to Vietnam to fight. Despite loving the music, I never had any interest in experimenting with the drugs. Beer, wine, and—occasionally—a martini or two are my drugs of choice and then only for medicinal purposes, as they say. Heck, I never even tried to smoke tobacco. Even as a child I couldn’t stand the smell of cigarette smoke to the point where it was never even really a temptation.
In any case, what provoked my original bit of musing was a post a few years ago by Martin Rundkvist, who wrote about Fear of Time Travel, where he imagines what it would be like for a modern person to be transported back in time:
First, imagine that you’re dropped into a foreign city with only the clothes you wear. No wallet, no hand bag, no money, no cell phone, no identification. Pretty scary, huh? But still, most of us would get out of the situation fairly easily. We would find the embassy of our country of origin, or if it were in another city, contact the local police and ask to use their phone. A few days later we would be home.
That’s not the scary scenario I rehearse. Imagine that you’re dropped into the city you live in with only the clothes you wear. No wallet, no hand bag, no money, no cell phone, no identification. And it’s 500 years ago. (Or for you colonial types, 300 years ago in one of your country’s first cities.)
It’s a fun thought experiment, with Martin pointing out that you would speak the language with what to the natives living at that time would seem a very strange and nearly incomprehensible accent. Think of how hard it is to understand the English spoken in Shakespeare’s plays, which is full of idioms, turns of phrase, and vocabulary peculiar to the time, and then just think about the number of words that we use that would be incomprehensible to, say, an American living in the Midwest, which at the time for where I live would have been ruled by the French as part of New France but mostly populated by indigenous tribes. So for purposes of the thought experiment, I’ll pick New York or Boston. I could also make like The Doctor and claim imagine that the TARDIS (or something else) had given me the ability to speak the native language and appear to be dressed like everyone else. Or I could imagine that I didn’t understand the language and had to learn it. Over time, it wouldn’t really matter. Here’s the part of Martin’s thought experiment that caught my eye:
Some might think that a well educated modern Westerner would soon become one of the sages of the age thanks to their superior technological and scientific knowledge. For one thing, it wouldn’t be hard for most of us to become the best doctor in the world of AD 1509 if knowledge was all it took. But I have a feeling that such knowledge would not be easily applied in a society that is completely unprepared for it, and not easily implemented in an environment where none of today’s infrastructure exists. And say that you’re actually a doctor or an engineer – how much could you achieve without access to any materials or tools invented in the past 500 years? I mean, I know the principles of nuclear fusion, aviation, antibiotics, vaccination and basic biochemistry, but don’t ask me to put them into practice starting from scratch!
Well, I am a physician and surgeon, and I don’t know if I could elevate myself to a sage of the age with my knowledge. The reason is that so much of what I do and have done in medicine relies on the technology and science of the time—this time, as in the decades between 1984 (when I entered medical school) and now. Let’s start with something very, very basic. I’m a surgeon. I try to cure or treat diseases by operating. Operating on a patient, however, is very difficult without reliable anesthesia, and inhalational anesthesia using ethyl ether wasn’t discovered and widely applied until the 1840s. Before that, there were various herbal anesthetics and hypnotics, natural drugs like opium extracts and later morphine, and even alcohol. While these may have sufficed for minor operations (barely), they were not at all sufficient for doing anything major, such as entering a major body cavity like the abdomen or chest.
That’s why, before anesthesia, surgeons had to be fast, and surgery was very bloody. Think of Abigail “Nabby” Adams Smith, the first born of our second President, John Adams. She was diagnosed with a malignant tumor of the breast and underwent a mastectomy without anesthesia, the gruesome details of which were described both in David McCollough’s biography John Adams and in Jim Olson’s Essay on Nabby Adams:
Nabby entered into the room as if dressed for a Sunday service. She was a proper woman and acted the part. The doctors were professionally attired in frock coats, with shirts and ties. Modesty demanded that Nabby unbutton only the top of her dress and slip it off her left shoulder, exposing the diseased breast but little else. She remained fully clothed. Since they knew nothing of bacteria in the early 1800s, there were no gloves or surgical masks, no need for Warren to scrub his hands or disinfect Nabby’s chest before the operation or cover his own hair. Warren had her sit down and lean back in a reclining chair. He belted her waist, legs, feet, and right arm to the chair and had her raise her left arm above her head so that the pectoralis major muscle would push the breast up. A physician took Nabby’s raised arm by the elbow and held it, while another stood behind her, pressing her shoulders and neck to the chair.
Warren then straddled Nabby’s knees, leaned over her semi-reclined body, and went to work. He took the two-pronged fork and thrust it deep into the breast. With his left hand, he held onto the fork and raised up on it, lifting the breast from the chest wall. He reached over for the large razor and started slicing into the base of the breast, moving from the middle of her chest toward her left side. When the breast was completely severed, Warren lifted it away from Nabby’s chest with the fork. But the tumor was larger and more widespread then [sic] he had anticipated. Hard knots of tumor could be felt in the lymph nodes under her left arm. He razored in there as well and pulled out nodes and tumor. Nabby grimaced and groaned, flinching and twisting in the chair, with blood staining her dress and Warren’s shirt and pants. Her hair matted in sweat. Abigail, William, and Caroline turned away from the gruesome struggle. To stop the bleeding, Warren pulled a red-hot spatula from the oven and applied it several times to the wound, cauterizing the worst bleeding points. With each touch, steamy wisps of smoke hissed into the air and filled the room with the distinct smell of burning flesh. Warren then sutured the wounds, bandaged them, stepped back from Nabby, and mercifully told her that it was over. The whole procedure had taken less than twenty-five minutes, but it took more than an hour to dress the wounds. Abigail and Caroline then went to the surgical chair and helped Nabby pull her dress back over her left shoulder as modesty demanded. The four surgeons remained astonished that she had endured pain so stoically.
Without effective anesthesia, I could do no better than these surgeons from 200 years ago or the surgeons from 300 years ago. In fact, I would probably do much worse, because I’m used to operating in a deliberate fashion, cauterizing individual blood vessels as I go. The reason surgery before science-based inhalational anesthesia was discovered was “cut and slash” was because it had to be. To do otherwise was to prolong what was quite literally the torture of the patient. I wasn’t trained to operate that way. I’m not used to operating that way, and there was no such thing as a Bovie electrocautery machine back then. It would all be scalpels, hot irons, and scissors–or, as in Nabby Adams’ case, razor blades and, in essence, a set of tongs to elevate the breast. It’s another reason why surgeons were frequently so fast in doing amputations that their assistants had to be careful not to let their fingers get in the way. As for more extensive operations, even with the anesthesia available in the latter half of the 1800s, a full 150 years after the time period Martin’s thought experiment envisions, there was no way to control respiration. Anesthesia was a delicate balance between not putting the patient so deep that he stopped breathing but putting him deep enough so that he wasn’t reacting overmuch to the surgical stimuli. Mechanical ventilators were an invention of the 20th century.
That’s just one example. There are numerous other tools, disciplines and examples of knowledge that a modern science-based surgeon depends upon in order to do his or her job, such as antibiotics and germ theory, pathology to identify what a patient has based on tissue samples, transfusions, and a wide variety of medications, to name a few. Then there’s diagnostic radiology. There would be no CTs or MRIs; there wouldn’t even be X-rays. Indeed, even something as simple as a suture would be a problem. The needles used 300 years ago were huge by today’s standards because of the difficulty of making small needles. The technology to do it was not widely available. Throughout history, needles were made of bone or metals such as silver, copper, or bronze, while sutures were made of cotton, flax, hemp, silk, or even tendons and nerves. There was a reason they called “catgut” suture catgut. Although cat gut suture was not made from the actual gut of cats, it was made from actual gut – connective tissue from intestines. (Actually, cat gut was pretty good suture and was still occasionally used 25 years ago when I first started my residency, mainly by the older surgeons.)
In any case, I think Martin’s right in that, without the infrastructure and scientific background being there, it would be very, very difficult for a surgeon of 2013 like me to recreate in the year 1713 much of anything that I do now, even if I were dropped into Boston among the most learned physicians of the age. No one there would have any idea of germ theory (and thus sterile technique, both of which were at least 150 years away), anesthesia (which was 130 years away), or much of basic physiology. Indeed, at that time, diseases were thought to be caused by imbalances in the four humors or miasmas, for the most part. If I were to try to explain the concepts that underlie the science-based medicine of today to the learned men of the time, assuming I could master the dialect of 300 years ago and find a way to describe the concepts to them, they’d assume I was either mad or a witch. It would be a good thing for me that the wave of witch hunts that swept through New England was pretty much over by the early 18th century.
There is one area that I can think of where a surgeon of 2010 might be able to translate some of his knowledge into 1710 and hope to have some influence. The first, of course, is sterile technique. It would not be that huge of an undertaking to sterilize instruments (although sterilizing sutures would be very problematic), either in flame or in alcohol. It would probably not be that huge a challenge to use alcohol, carbolic acid, or some other compound to clean the operative field, the patient’s skin, and one’s hands. (Given the lack of latex or rubber gloves, I’m not sure if it would be possible not to operate with my bare hands, as surgeons of the time did–and in fact continued to do until the late 1800s and beyond). In other words, I could be Joseph Lister 150 years before Lister showed the benefits of antisepsis, publishing his seminal paper on the benefits of antisepsis in 1867.
Come to think of it, I could potentially be Louis Pasteur, again 150 years before Pasteur did much of his work. At the very least, I could figure out how to replicate his experiments disproving abiogenesis and to develop Pasteurization. Of course, convincing the world of 1710 of the validity of these ideas would be even harder than it was for Louis Pasteur to convince his contemporaries of germ theory, because far less of the ground work would have been laid. SBM builds on the science of the times incrementally, and if the requisite prior discoveries haven’t been made yet it’s hard for new science-based modalities to take hold. Lister’s discovery, for instance, didn’t take hold in the US until over a decade after it took hold in Germany because of resistance to the germ theory of disease.
In fact, it brings up an issue that should demonstrate just how hard it would be to convince the physicians and surgeons of 1713 of our knowledge. Consider something as simple as blood pressure. Although the ancient Egyptians knew enough to palpate pulses, the very first measurements of blood pressure (more accurately, pulse pressure) were not made until 1733 (twenty years before I’d be dropped into New England) by Stephen Hales, who measured the blood pressure of a horse. It was not until 1855 when the first sphygmomanometer was devised by Vierordt of Tubingen, an instrument called at the time the sphygmograph, which was considerably improved upon by Etienne Jules Marey in 1860. Before this, it was not possible to measure arterial blood pressure other than under surgical conditions using a canula inserted directly into an artery. Finally, the modern version of the sphygmomanometer was invented by Samuel Siegfried Karl Ritter von Basch in 1881, but Italian physician Scipione Riva-Rocci improved upon it by producing a more easily-used version in 1896. After that, Harvey Cushing discovered the device during a visit to Italy in 1901 and popularized its use in the U.S. after he returned home. What all this means is that the routine measuring of blood pressure as a “vital sign” in virtually all patients did not become truly routine until 100 years ago. Even more amazing, it was still 20 years before Russian physician Nikolai Korotkov popularized the use of the device to measure diastolic blood pressure as well, meaning that the systolic/diastolic blood pressure ratio that we’re all familiar with didn’t become routine practice until the 1920s.
Hmmm. Maybe I could be Harvey Cushing 200 years before Harvey Cushing was in his prime.
In any case, if there’s one thing this little thought experiment has done for me, it’s to make me realize how much of what I do depends on hundreds of years of history and science and any achievements I may have in my career rest squarely on the shoulders of giants. I’ll take another example from my very own specialty, the “god” of modern surgery William Stewart Halsted, who practiced surgery from the late 1870s until his death in 1922. During that time, he pioneered so many surgical advancements that it’s amazing that one person came up with so many ideas and pushed the envelope so much.
In imagining myself 300 years ago trying to replicate such advances more than 150 years sooner than Halsted, I have to think about the sorts of resistance he encountered. For example, when he was working at Bellevue Hospital in New York, Halsted saw Joseph Lister present his discoveries. Inspired by Lister, Halsted tried to convince hospital administrators to construct a sterile operating room. The hospital refused, because it would have been very expensive and was considered at the time to be “wildly innovative.” (The administrators, I suspect, didn’t mean that in a good way.) So Halsted used $10,000 of his own funds to erect a tent on hospital grounds. He equipped it with maple floors, gas lights, and sterilization facilities. Basically, it was the first planned sterile operating environment in the U.S., possibly in medical history. Ironically, Halsted had also pioneered the use of sterile latex operating gloves because his fiancée (she was his nurse at the time) complained that the antiseptic mercuric bichloride had caused dermatitis on her hands. Given how much difficulty physicians and scientists like Halsted and, before him, Ignaz Semmelweis and Louis Pasteur had convincing a scientific world that was primed for their discoveries, imagine how much difficulty I would have convincing the physicians and scientists of 1713 of my discoveries. I can’t help but think that if I were to be dropped in Europe in 1513 (Martin’s original idea of traveling 500 years back in time), I might be introduced to the Tribunal of the Holy Office of the Inquisition, depending upon where in Europe I ended up.
Of course, Halsted was also often rather adventurous (some might say overly so) in his achievements, for example:
In 1881, while visiting the family home in Albany, New York, he performed the first emergency blood transfusion, drawing his own blood and injecting it into his sister’s bloodstream when he found her passed out from a postpartum hemorrhage. The next year he performed emergency gall bladder surgery on his mother, performing the operation on her kitchen table — administering ether to knock her out, sanitizing his hands and equipment by dipping them in carbolic acid, then slicing into her abdomen and gall bladder, draining vast amounts of pus and removing seven stones — the first recorded surgery to remove gallstones.
Obviously, both of these emergencies could have ended disastrously. Halsted’s sister could easily have died from a transfusion reaction, given that virtually nothing was known at the time about immunology or blood typing, and his mother could easily have died due to her home surgery, although given the lack of antibiotics she would almost certainly have died without treatment. No doubt Halsted knew that.
How I treat breast cancer also depends upon hundreds of years of painfully-gained advances in scientific knowledge and is (usually) the best that can be achieved at the time. This inevitably brings me back to the example of Halsted. Halsted performed the first radical mastectomy in 1882. It was a long, extensive, and bloody operation for the time. In talks on the history of breast cancer surgery that I have given, I like to point out that, early on, Halsted’s surgery was viewed as butchery. However, as his survival data matured and it turned out that he produced survival rates better than other surgeons operating at the time, his method became the standard of care for the next 80 years. Yes, it was brutal surgery, but it made sense at the time. When Halsted operated, there was no chemotherapy. Even though Emil Grubbe had demonstrated the first use of radiation to treat cancer in 1896, radiation therapy was crude and the concept of adjuvant radiation therapy was decades away. If breast cancer was going to be cured, surgery alone would have to do it, and, given the scientific understanding of cancer at the time, it made sense to perform a radical resection of the breast and all lymph nodes to which the cancer might drain. Later scientific discoveries and developments rendered such radical surgery unnecessary, but, although it can be argued that such surgery persisted as the standard of care too long after other, less disfiguring, surgery became possible, it makes no sense to criticize the surgery as barbaric when at the time it was the best available and was consistent with the then-current science.
Finally, this fun little thought experiment brings me back to the present through the lens of history. Critics of SBM, in particular those who advocate what we like to call unscientific medicine or pseudomedicine, often make the claim that today’s treatments of cancer will, 50 or 100 years from now, be considered barbaric and unscientific. This sort of idea is sometimes expressed in pop culture as well. Indeed, one is reminded of a scene in Star Trek IV: The Voyage Home (whose plot, not coincidentally, involves the crew of the Enterprise time traveling back to San Francisco in 1986). In that scene, some of the crew finds itself in a hospital of 1986, and Dr. Leonard “Bones” McCoy is horrified at dialysis, which he likens to the Dark Ages:
Later in the scene, he likens a debate between two residents about radical chemotherapy to the “goddamned Spanish Inquisition.”
Did I ever mention, by the way, that Dr. McCoy was one of my inspirations for becoming a doctor?
Of course, this movie scene is a perfect way to end this post. In ST:IV, the time difference is 300 years, the crew of the Enterprise having traveled back from Star Date 8390, which corresponds to the year 2286, back to the year 1986. Perfect! This is pretty much the same as my traveling 300 years back in time to 1713, at least enough for my purposes. I’ve tried to give you a flavor of how different that world was in terms of medicine and how out of place my skills would be there. All that I know about medicine and all of my skills as a surgeon and scientist rest on the advancements of anatomy, chemistry, physiology, and science that have been painstakingly discovered, mostly over the last 300 years—actually, mostly over the last 150 years. So it’s not at all unreasonable to guess that to a fictional doctor from nearly 300 years from now our medicine would seem as primitive as the medicine of 1713 does to a physician like me from today. However, disparaging such medicine as primitive and brutal is misguided. The medicine of 1713 was brutal and primitive because the science didn’t yet exist to make it otherwise. A physician traveling back to that time would discover just how true that is and how difficult it would be to convince his peers of the time of how much better his science and treatments are.
In other words, even if it is true that the science-based medicine of 2313 would be as far ahead of the science-based medicine of 2013 as the SBM of 2013 is compared to the best of 1713 (or the fictional SBM of 2286 is compared to that of 1986), that’s beside the point. We don’t know the science that will inform the medical advancements of the next 300 years any more than physicians of 1713 had a glimmer of an idea of the advances and discoveries that would be made over the next 300 years from them. SBM is always changing and adapting in response to new scientific discoveries. The process is often slower and messier than we would like, but it does continue. The SBM of today will give way as new discoveries are made, and it is not at all shameful if medicine in even 50 years is very different than it is now. After all, medicine now is much different in so many ways from what it was in 1963, and that doesn’t make the physicians of 1963 incompetent. Like us, they were products of their time.
Finally, it occurs to me that the way medicine was practiced 300 years ago here in the colonies bears a lot of resemblance to many “alternative” therapies. Medicines back then were virtually all derived from herbs or animal products, often the herbs or animal products themselves, unpurified. Germ theory was at least 150 years in the future, and disease was thought to be due to things that very much resemble alt-med’s concepts of disturbances in the flow of qi. While I might be very much a fish out of water in 1710 as far as trying to practice medicine and surgery, I suspect many “alternative” medicine practitioners would not be, and that’s the difference. SBM changes substantively as science advances. In “complementary and alternative medicine,” (CAM), the ideas remain the same, and only the names change.

The Forerunners of EBM
Harriet Hall on June 5, 2012
Link
The term “evidence-based medicine” first appeared in the medical literature in 1992. It quickly became popular and developed into a systematic enterprise. A book by Ulrich Tröhler To Improve the Evidence of Medicine: The 18th century British origins of a critical approach argues that its roots go back to the 1700s in Scotland and England. An e-mail correspondent recommended it to me. Can’t remember who, but I would like to thank him.
Francis Bacon (1561-1626) differentiated between “ordinary experience” (chance observations) and more objective “ordered experience” (methodological observations). Both of these involved empirical knowledge. It’s hard to get back into the mindset of his time, when most physicians rejected empiricism as the sphere of quacks and surgeons. Tröhler helps us understand why they did:
…since antiquity, the mark of distinction of a learned man had been the certainty of his knowledge. A doctor knew — he did not need to test his kind of knowledge empirically because this would imply acknowledgement of uncertainty.
Ancient wisdom, tradition, and dogmatism reigned supreme, in medicine as in religion. Rationalism based on accepted theory was sufficient: doctors could reason their way with no need for testing.
Surgeons were perhaps the first to empirically compare the effects of different treatments. In 1536, Paré tried a raw onion poultice on a burn case, on the advice of an old country woman, and found that the onion-treated areas of skin were free of the blisters seen on the conventionally treated areas of skin. He also found that treatment with boiling oil left wounded soldiers in agony, while an ointment containing turpentine did not. He wasn’t finding out what worked so much as finding out which remedies did the most harm.
The British Navy was an ideal place to experiment: the same doctor controlled the treatment of everyone on a ship and he had a captive group of subjects for months on end. James Lind’s famous scurvy experiment took place in 1747, but he didn’t invent his method out of the blue. He was influenced by other thinkers of the time. It took 42 years for his discovery to be implemented, but Tröhler explains why it was logical that it took so long. No one had any idea of vitamins. Scurvy was envisioned to be a “putrid” disease. The theories of the time implicated the fermentative qualities of vegetables. Unfermented malt (“the wort”) was thought to have the same properties as citrus fruits, and some patients improved on wort, likely because it supplied missing B-vitamins that were the cause of some of the patients’ symptoms. Wort was cheaper and easier to supply than lemons. They continued to try other things, like sauerkraut, mustard, vinegar, sugar, and molasses. An early effort to supply ships with bottled lemon juice failed because it was boiled, destroying the vitamin C. When you try to re-create the thinking processes of the time, 42 years of resistance doesn’t look so shocking.
The story of fever is enlightening. They thought in terms of bad air; microbes were yet to be discovered. They lumped all fevers together into one diagnostic category. Cinchona bark is an effective treatment for malaria, but when they used it on other kinds of fevers, they naturally got varied results.
We laugh at bloodletting today, but the 4-humor theory was well established and generally believed, and early trials of bloodletting were equivocal: some showed benefit, others didn’t. It might have seemed a simple matter to test it and discard it, but the situation was much more complicated in practice. Those who favored it could actually point to convincing empirical evidence. Battles raged for decades before it was abandoned.
Early controlled trials were flawed. They mostly compared two treatments, with no placebo or untreated group. Unrecognized factors affected results. The placebo-controlled, randomized trial wasn’t invented until well into the 20th century. But at least they were trying.
The idea of applying “arithmetic” to medicine was a new one. They were used to writing up a detailed case report of one successful treatment so that other doctors might read all the details and notice some common feature with their own patients. Reporting failures was not the custom: they didn’t see any point in reporting about something that didn’t work. Reporting numbers of successes and failures for a group of patients was a new idea that many found objectionable, because they favored individual evaluation and treatment. Tröhler does a good job of describing their thought processes and the oh-so-gradual groping towards better ways of “improving the evidence of medicine.”
One of my personal heroes is William Withering, for his scientific rigor and sensible humility. He wrote his Account of the Foxglove in 1787. He had observed an apparent response to the folk remedy foxglove for dropsy (congestive heart failure) and he set out to test it systematically. He described every case in detail, including dosage, method of preparation and all the failures. With the striking results he got, all too many modern doctors would have contacted the media to boast that they had discovered an effective remedy. Withering limited himself to saying that if others confirmed his observations, the plant might become a useful part of the standard treatment of heart disease. Others did confirm, and the plant did become a useful part of standard treatment. The active ingredient was isolated and improved upon, and Digoxin is still widely used today. Interestingly, it has fallen out of favor for its original use in congestive heart failure: although it reduces hospitalizations, it has been shown not to reduce mortality and we have better drugs that do. It is still commonly used for certain heart arrhythmias, although it is not the drug of first choice. Medicine evolves in response to better evidence.
I was surprised to see how large a role military medicine played in testing treatments and accumulating numerical evidence for everything from amputations to syphilis. (No need for informed consent, of course.) Outstanding work was done by thoughtful doctors outside the mainstream establishment of prestigious upper-class physicians. They worked on their own initiative, without monetary reward, to put the received wisdom of medical authority to the test. We owe those pioneers a great debt of gratitude.
One thing that struck me was the recurrent theme of “inflammation” as the alleged cause of fevers and practically everything else. It was a simplistic explanation that was wrong, and it reminded me of today’s simplistic claims for anti-inflammatory diets and antioxidant supplements.
Tröhler’s book is no literary masterpiece and its recital of details becomes frankly boring at times, but it’s full of tidbits of interesting information for history buffs and of lucid explanations for those who want to understand the evolution in thinking processes that led, slowly but surely, to modern scientific medicine. The book is not easy to find. It was published in 2000 by the Royal College of Physicians of Edinburgh. It is out of print and not available in used bookstores. The only way to obtain it is through interlibrary loan. My local public library located a copy at a university library in Ohio and shipped it all the way to Washington State for me to read. Didn’t take long, and didn’t cost me a penny. I found the book enlightening, and I feel more charitable towards our predecessors than I once did. 

The Need for Science-Based Medicine
The mission of SBM continues.
Steven Novella on December 15, 2021
Link

We are coming to the end of the 14th year of Science-Based Medicine (SBM), having published over 4,000 articles. The end of another year is always a good time to look back and see how we are doing. Of course, it’s a difficult thing to measure – what would be different had we not been around the last 14 years?
Our primary mission is to be a forum where professionals, the media, and public can go for a rigorously science-based perspective about popular treatments and claims that tend to avoid scientific scrutiny, or who distort the scientific process to their advantage. At the same time we acknowledge that science communication is not just about giving information, or countering misinformation. We need to cultivate the skills to distinguish science from pseudoscience, and to tell reliable from unreliable sources of information. Further, misinformation is often woven into a comforting or compelling narrative, often with a specific goal (usually to sell something). SBM needs to provide a counter-narrative, one based on science and reason.
In addition to helping people make better medical decisions for themselves, we hope to influence society as a whole. This means pushing scientific and medical institutions to remain clear-eyed on what SBM is and to hold the line against the creep of pseudoscience into medicine. There are well-funded forces at work constantly trying to insinuate nonsense into the institutions of medicine, and these institutions are not as well defended as we might hope. Scientists and professionals generally don’t understand how pseudoscience works – that is where we try to fill the gap.
There are also constant battles of regulation to be fought, again with well-funded groups pushing to ease regulations that are designed to protect consumers from health fraud. They provide constant pressure to lower standards, or remove them entirely, and to move us toward a world where the health consumer is almost entirely on their own. Without rules, fraud and deception can thrive.
Finally we are constantly questioning and refining how to optimize SBM itself. It’s a complex question, how best to connect scientific evidence to medical practice. In fact it’s a science unto itself. I have learned a tremendous amount myself over the last 14 years. At the beginning of this project I had never heard the terms “P-hacking” or “citation bias”. Nor did I fully understand how so-called “pragmatic” studies were abused to make efficacy claims they were not designed to test. But these nuances are exploited by pseudoscientists with an agenda, and they crop up with well-meaning professionals who simply need to do better.
In the last two years the pandemic has also brought our current situation into sharp focus. It has been a “battle royale” for all the forces of science and pseudoscience within not just medicine but the broader society. All the factors, from social media to mainstream outlets, government bodies, and medical institutions, are involved as we try to find our way through a deadly pandemic. Everyone, now, has a strong opinion about vaccines and those who refuse them, and the “what’s the harm” argument about tolerating magical thinking in medicine is getting more obviously flimsy. We are all living through the harm.
This is also the time of year when we ask for your support. SBM is not just a blog or an approach to medicine, it is a community of people who want to improve the world, to push back against exploitation and health fraud, or who just want to be better informed and more critical thinkers. All of the writers and editors at SBM donate their time without any compensation, and it is absolutely our pleasure and a labor of love. But we recognize that we are pushing back on many fronts, against many well-funded interests. At the very least, we need to keep the lights on so we can continue our work.
If it’s within your means, please consider becoming an SBM patron, or making a one-time donation (links are on the homepage). For any professionals out there, also please consider contributing to SBM. The greater variety of voices we have the better.
        ◦ The mission of SBM will never be over. This is not the kind of goal where we can declare victory and stop. Society will always need people advocating for consumer protection, and for high standards of science in medicine, not for any personal gain but simply because it needs to be done. And there will always be those who try to game the system to their own advantage, or are simply misguided or misinformed. Like science itself, SBM is a process, not a conclusion or destination. We’ll be here in 2022 and beyond – thanks for being there with us.

In which I steal a title: Is medicine a scientific enterprise?
David Gorski on June 22, 2015
Link


A week ago, I attended the Center For Inquiry Reason for Change Conference, where I participated in a panel on—what else?—alternative medicine with—who else?—Harriet Hall and our fearless leader Steve Novella. Before the panel, we all gave brief talks on areas that we consider important. As you might expect, I chose to give a brief introduction to what I like to call “quackademic medicine,” defined as the pseudoscientific medicine being practiced and studied in academic medical centers. As I like to do in order to drive the point home about just how bad it’s become, I chose a couple of truly egregious examples of just how much quackery has infiltrated medical academia. First, I mentioned how the Cleveland Clinic has embraced reiki, which, as I’ve described many times before, is in reality faith healing that substitutes Eastern mysticism for Christian beliefs. Although I could have buried the audience in examples, the other example I happened to choose was this:
#Energytherapies offer support in healing for #cancersurvivors #LivingWithCancer http://t.co/AUyX8cLj4w pic.twitter.com/0N30G7VL65
— Mayo Clinic (@MayoClinic) June 7, 2015
Yes, that is exactly what you think it is. It’s the official Twitter account of the Mayo Clinic promoting “energy therapies,” of which reiki is one of the most popular varieties. Basically, these are “therapies” in which it is claimed that the practitioner can either (1) manipulate the “life energy” fields of the patient (e.g., healing touch) or (2) channel “healing energy” into the patient from a source (e.g., reiki). And here was the Mayo Clinic promoting this magical mystical nonsense, linking to an article on its official website entitled “Energy Therapies Offer Support in Healing for Cancer Survivors“: 
Reiki assists in balancing your physical, mental, emotional and spiritual well-being. The practitioner works with your body’s energy field to bring the energy into balance and create a sense of calm and peace. Healing Touch uses the same principles with the goal of balancing the energy in and around your body. With Healing Touch, the practitioner also uses hand movements on or slightly above the body to clear and balance the energy fields around the person.
Not surprisingly, this promotion of mysticism (and, yes, “healing touch” and reiki are nothing but pure mysticism) provoked a reaction. One of these was from our very own Clay Jones:
@MayoClinic You’re an embarrassment to our profession. — Clay Jones (@skepticpedi) June 7, 2015
Grant Ritchey, a dentist who has blogged for us from time to time in the past, also chimed in:
.@drflanders @skepticpedi @MayoClinic Why would the esteemed Mayo Clinic stoop to unproven quackery? Marketing? Looking at the bottom line?
— Grant Ritchey Jr. (@SkepticalDDS) June 7, 2015
They weren’t alone in protesting, either. I merely chose to highlight their Tweets because, of course, they’re part of the SBM family. It was at this point that a physician named Michel Accad, MD, a cardiologist in San Francisco who has his own blog and has contributed to KevinMD.com, chimed in. I didn’t recall ever having encountered him before, and a search of the blog didn’t find any mention. However, in a nutshell, he expressed one common complaint about science-based medicine on Twitter:
@skepticpedi @drflanders @MayoClinic Patient finds benefit; no apparent harm; who am I to judge…? — Michel Accad MD (@michelaccad) June 8, 2015
This is an attitude that is at the very heart of how many misunderstand what Science-Based Medicine is. It is also a pernicious attitude that leads so many physicians to become and remain what Val Jones once so famously called them, “shruggies.” They know that modalities like reiki and the rest of “energy medicine,” homeopathy, acupuncture and the vast majority of traditional Chinese medicine, and much of what is being “integrated” into medicine under the rubric of “integrative medicine” range from, at best, being at best scientifically unsupported to, at worst, being pseudoscientific or mystical nonsense, but they do nothing. They do not object. They do not complain. They shrug their shoulders.
If only Dr. Accad had simply expressed no more than a “shruggie” attitude. Instead, he decided to write a post entitled “Is medicine a scientific enterprise?” It’s a post full of straw man characterizations of SBM, appeals to “science isn’t enough,” and attacks on what he calls the “Flexnerian error of scientific medicine.” Because Dr. Accad perfectly encapsulates fallacious objections to SBM, I thought his post, which quotes several of those Tweets above as a jumping off point, would itself represent a good jumping off point to correct some of the misconceptions about SBM that are out there.
Is medicine a scientific enterprise?
Let’s start out with the title of Dr. Accad’s post, “Is medicine a scientific enterprise?” I counter that this in itself is the Burning Man-sized straw man around which all of Dr. Accad’s other straw men gather as it burns. No one argues that medicine is purely a scientific enterprise, which is the argument that Dr. Accad is countering. Even we here at SBM do not. None of this stops Dr. Accad from forging bravely ahead into the burning straw:
The outrage, of course, reflects the belief held by some that medical care should be “scientific” and purged of all “nonsense.” This idea is particularly popular among enthusiastic champions of a peculiar view of science as the sole and omnipotent purveyor of all real knowledge. Some of those involved in the tiff were undoubtedly of that persuasion.
But apart from holding sway in the minds of Richard Dawkins devotees, the notion that medicine should be a scientific undertaking pervades, to varying degrees, the entire health care community. After all, it is on the basis of this idea that Abraham Flexner boldly constructed the report which gave birth to our health care system a little over a century ago.
It was at this point that I was tempted to respond to Dr. Accad: “You say that as though it were a bad thing,” but I resisted the temptation. Oh, wait. No I didn’t. I actually did use that line when discussing this at the CFI conference. Never mind. Sarcasm aside (seriously, what does Richard Dawkins have to do with medicine, other than as part of a two-part documentary he did for the BBC back in 2007?), there’s more of the straw man. Specifically, the straw man is the implication that supporters of SBM (like us) think that doctors should all be Mr. Spock rather than Dr. McCoy and approach patients with nothing more than pure science and logic. It’s a caricature of Richard Dawkins, and it’s a caricature of SBM and its advocates, pure and simple—and an exceedingly lazy one at that—using Richard Dawkins as an all-purpose bogeyman for “militant” advocates of science because he has been such a staunch advocate of science and has also spoken out against alternative medicine in the past. Never mind that advocating science-based medicine is certainly not what Dawkins is best known for.
Yes, we of “that persuasion” do argue that the best medical care should be based in science, because the fruits of applying science to medicine—something that really didn’t happen in a big way until around 150 years ago—are there for all to see. For example, infectious diseases are prevented by vaccines and cured by antibiotics. Infant mortality has plummeted, largely due to decreases in death due to infectious diseases that in Abraham Lincoln’s time it was not uncommon for a mother to bury four or more of her children. Even in 1900, one in seven children did not make it past age one. Life expectancy has increased markedly from that same time, when it was 46 for males and 48 for females. Now the same figures are 76 and 81, while death rates from heart disease and cancer have been falling. Though, of course, it’s certainly not just science in medicine but also science in public health that brought clean water and better sanitation into cities that were cesspools of disease in the 19th century. There are many other examples of how the application of science to medicine has benefited humanity enormously.
But what does “based in science” mean? There’s the rub, as they say. I will explain in a moment (hint: it doesn’t mean what Dr. Accad apparently thinks it means), but first let’s look at a bit more of Dr. Accad’s complaint:
Now, the idea that medicine should be a scientific enterprise–even to the slightest degree–is an erroneous idea. Medicine itself cannot be viewed as scientific for the simple reason that the aim of science is to acquire knowledge, whereas the aim of medicine is to heal. These are two distinct ends. Furthermore, a scientific enterprise is best carried out with dispassion: observation and experimentation. Healing, on the other hand, is best accomplished through personal involvement: caring.
Can you say “false dichotomy”? Sure, I knew you could. Of course, there’s a reason why Steve Novella called this blog “Science-Based Medicine” and not “Scientific Medicine” or “Medical Science” or some other variant thereof. It’s because the central thesis behind this blog is that the best medicine is based on science, not that medicine itself must be pure science. As Steve put it in the very first post to appear in SBM way back in 2008:
The philosophy of this blog, at its core, is simple: Safe and effective health care is critical to everyone’s quality of life; so much so that it is generally considered a basic human right. The best method for determining which interventions and health products are safe and effective is, without question, good science. Therefore it is in everyone’s best interest for health care to be systematically evaluated by the best science available.
And:
This is why the authors of this blog strongly advocate for science based medicine – the use of the best scientific evidence available, in the light of our cumulative scientific knowledge from all relevant disciplines, in evaluating health claims, practices, and products.
In other words, no, we do not claim that science is the “omnipotent purveyor of all real knowledge.” We do, however, claim that science is the best method of evaluating which medical claims, understandings, and treatments are safe and effective and which are not. In that sense, science is the best method to evaluate health claims. Certainly, no one, least of all Dr. Accad, has proposed a better method. Indeed, because medicine involves treating patients, an activity that involves taking into account each patient’s unique situation and values, treating patients can never be completely scientific. It’s not as though we haven’t discussed this many times before here. No one expects Dr. Accad to have read those posts or even to be aware of the existence of this particular blog, but I do expect him to think a little more deeply than 140-character Twitter posts. For instance, I know Clay Jones and Grant Ritchey understand this, but it’s the sort of thing that’s hard to say in 140 characters. Chastising the Mayo Clinic for promoting mystical faith healing as a valid treatment modality, however, is easily accomplished, and that’s what they did.
Unfortunately, Dr. Accad’s thoughts on the matter did not become clearer once he was freed of the tight constraints of Twitter, for he is positing a false dichotomy. Again, he seems to be arguing that one can’t apply science to medicine without losing the “human touch,” to which I respond, quite bluntly: What a load of fetid dingo’s kidneys! There is nothing—I repeat nothing—in science-based medicine that excludes caring for patients as a human being or being what so many proponents of “integrative medicine” like to refer to as a “healer.” Nothing! Let’s just put it this way: You can be an empathetic, caring doctor who uses nothing but science-based medicine, and you can be a tone-deaf, uncaring practitioner who incorporates reiki, homeopathy, and all manner of pseudoscience into his practice. The two (practicing science-based medicine versus the empathy-based and potentially-unscientific medicine that Dr. Accad seems to prefer and being an empathetic, caring physician) are not related other than in that those practicing unscientific medicine tend to have to come across as more empathetic and caring because that’s all they have other than placebo medicine!
Besides, in the end, effectiveness is what matters. If forced to choose between an empathetic “healer” who holds my hand as I die of sepsis and a wooden, distant doctor who saves my posterior with just the proper application of antibiotics and drainage of pus, guess which one I’ll pick. Sure, it would be nice to have both, and fairly often a patient can have both, but, again, the application of the correct science-based treatment that is effective trumps empathy for most people. Don’t get me wrong, empathy and caring are very important, but they offer little in terms of actual physical healing if the medicine being practiced has not been shown by science to be safe and effective. Without science-based medicine, a doctor can only be as effective as doctors of hundreds or thousands of years ago; i.e., not very or possibly even worse than the disease. Such was the state of medicine, as full of empathetic “healers” as it was, for thousands of years before the evil Abraham Flexner made it all scientific a hundred years ago.
Of course, what Flexner actually did went beyond emphasizing science in the curriculum of medical schools. If you read the actual report, you’ll note that in its introduction Flexner decries the “overproduction of ill-trained” physicians by a “large number of commercial schools, sustained in many cases by advertising methods through which a mass of unprepared youth is drawn out of industrial occupations into the study of medicine.” In the end, Flexner made the following recommendations:
    • Reduce the number of medical schools (from 155 to 31) and poorly trained physicians
    • Increase the prerequisites to enter medical training
    • Train physicians to practice in a scientific manner and engage medical faculty in research
    • Give medical schools control of clinical instruction in hospitals
    • Strengthen state regulation of medical licensure
Again, one wonders what Dr. Accad thinks is so bad about these recommendations:
Of course, this is not to say that scientific inquiry cannot inform doctors on the proper course of action. It certainly can, should, and does.
Which is all science-based medicine is, the idea that science must inform doctors on the proper course of action. That’s all Steve argued in the very first post on SBM and all we continue to argue: for a single science-based standard for evaluating medical claims. Unfortunately, Dr. Accad goes off the deep end right after:
But scientific inquiry can only be subordinate to medical care. It is because one cares for the patient that one seeks the best material ways to cure or treat the body, and scientific knowledge provides valuable information in that regard.
It was at this point that I as scratching my head with a hearty “WTF is this guy talking about?” What on earth is he referring to about scientific inquiry being subordinate to medical care? After all, when it comes to clinical trials (i.e., scientific inquiry), whether Dr. Accad realizes it or not, scientific inquiry is already subordinate to medical care. Medical ethics demands no less. That’s why it’s unethical to do a randomized controlled clinical trial of vaccinated versus unvaccinated children. It would do harm to the children in the unvaccinated arm, who would not receive the proper medical treatment to reduce their risk of deadly disease. That’s why we’re forced to rely on epidemiological studies to study vaccines and that’s why the only randomized controlled clinical trials of vaccines these days involve adding a vaccine to the existing schedule versus placebo, so that both experimental groups get at least the standard of care. That’s why cancer chemotherapy trials testing a new drug versus only placebo are becoming rarer and rarer; instead such trials are usually standard of care plus new drug versus standard of care plus placebo. Again, both groups get at least the standard of care. That’s what the entire concept of clinical equipoise is about, namely that there has to be genuine uncertainty about which treatment is better before a clinical trial (scientific inquiry) is warranted. In that way, scientific inquiry is and will always be subordinate to medical care.
But that’s not what Dr. Accad is talking about, as you will see.
Medicine and the nature of science
Dr. Accad’s little diatribe against “scientism” in medicine—which, let’s face it, is what he’s railing against, although he doesn’t actually use the word—would be bad enough, given the straw men torched by just the halfway point. In fact, I’m surprised he didn’t use the word, given that his description of critics of “complementary and alternative medicine” (CAM) and “integrative medicine” as believing that science is the “omnipotent purveyor of all real knowledge” is as good a definition of scientism when used as a pejorative (as it almost always is) by advocates of pseudoscience as I’ve ever seen. Unfortunately, Dr. Accad’s understanding of science is—shall we say?—not exactly robust:
First, we should recognize that biomedical science is only scientific in a limited way. When, at the dawn of the modern era, science separated itself from philosophy to take on a decidedly empirical cloak, the human observer could no longer properly be the subject of scientific inquiry, except in an indirect manner. And where physics and chemistry have been able to uncover “laws” of nature, biomedical science generally limits itself to making tentative, statistical predictions on human data aggregates—populations.
Causality in physics (notwithstanding its Humean objections) is not the same as causality in biology, and is even less related to causality in human affairs. Medical care is a pursuit of health for the good of the [sic] another human being, an individual person. That person exists in and expresses a contextualized reality: Mrs. Jones is not a 76-year old woman with cancer, except as a shorthand identification. Mrs. Jones is a being of a rational, self-determining nature and, strictly-speaking, incomparable to any other 76-year-old woman with the same cancer. The scientifically “proven” benefit or ineffectiveness of treatment X is never proven in the case of Mrs. Jones.
Biomedical sciences aren’t as scientific as physics? At the risk of repeating myself, what a load of fetid dingos’ kidneys! Biomedical sciences are based on chemistry, which is based on physics, because all life is chemistry! Just because there is considerably more variability in repeated observations in biology and biomedical sciences than there is in physics does not make biology any less scientific than physics. As for “laws” of nature, scientific laws are generally of the sort that can be reduced to simple equations, such as E=mc2. In contrast, a scientific theory usually seeks to synthesize a body of evidence or observations of a particular phenomenon. It is, as Jacob Silverman put it, a “grander, testable statement about how nature operates” that, although not reducible to a pithy statement or equation, “does represent something fundamental about how nature works.”
And guess what? There are theories in biology and medicine that were derived from those very “tentative statistical predictions” on populations. There’s the theory of evolution, for example, which infuses all biological science, as well as phenomenon in medicine such as the evolution of bacterial resistance to antibiotics and tumor cell resistance to chemotherapy. There’s the germ theory of disease (of course), a major advancement in the late 1800s from which flowed much of the initial success of scientific medicine. Over time other theories emerged, such as the theories governing molecular biology, which lead to theories about the genetic causes of diseases ranging from inborn errors of metabolism to cancer.
Nor does SBM conflict with patient autonomy, as Dr. Accad’s example of Mrs. Jones, the 76 year old woman with cancer, implies. Indeed, as I’ve argued time and time again, SBM is necessary for true patient autonomy. It is because Mrs. Jones is a rational, self-determining human being that she deserves the best information we as physicians have available about her disease, the treatments available, the risks and benefits of these treatments, and the pros and cons of these treatments relative to each other as the basis to make a collaborative decision with her physician on how to treat her disease. Without that information, she can never provide true informed consent, only misinformed consent (as I like to put it). And, yes, Dr. Accad, science really is the best means of answering these questions in medicine. (If a better means exists, Dr. Accad certainly hasn’t suggested it—just like every other doctor defending quackery like reiki on the basis of “science isn’t enough.”) It is true that Mrs. Jones brings her own values and unique situation to the interaction with her physician. For instance, she might have decided that at 76 she values quality of life over remaining quantity of life and therefore doesn’t want to suffer harsh side effects or take major risks. Alternatively, she might desperately want to live long enough to see her grandchild marry and be willing to take chances to see that happen, if medicine can make it possible. However, she cannot properly incorporate her life values and experience into the question if someone is telling her that magic will do her good, particularly if that person is wearing a white coat and is speaking with the authority society gives physicians for their knowledge of medicine.
Now, those of you sympathetic to SBM, be sure to put down your drink if you have one and swallow whatever it is you’re drinking. I don’t want to be responsible for any ruined keyboards. You have been warned.
Dr. Accad argues next:
Secondly, to circumscribe medical care inside the realm of science limits the autonomy of the patient. Perforce, “scientific medicine” in the Flexnerian sense separates the physician from the patient, because the latter becomes an object in (or a subject of) the scientific enterprise, and therefore, at some level, must be deemed incapable of judging the value of the care received: no one asks the falling apple if it would prefer to be considered under the law of universal gravitation or under the general theory of relativity.
The Flexnerian notion of scientific medicine, then, brings to the fore the “information asymmetry” in the doctor-patient relationship and justifies State intervention by way of licensing laws. In turn, licensing laws give credence to and materialize this asymmetry. Patients, as object of scientific medicine, can no longer freely choose their care as the State intervenes to ensure safety and efficacy according to objective, scientific norms.
The claim that “Flexnerian” medicine causes the patient to become an object of the scientific enterprise and incapable of judging the value of the care received is a straw man so massive that, were it to be burned, not only could it be seen from space but the smoke it would produce would single-handedly accelerate global warming by several decades. In fact, it is science that has shown that human perceptions alone are unreliable and that the various effects that are often bundled together as shorthand under the term “placebo” effects do mean that patients will frequently feel better subjectively after an intervention that can’t do anything physiological, such as homeopathy. I would argue that it is the job of a physician to do better than that whenever possible, as placebo effects are an incredibly low standard to hold ourselves to.
Patient autonomy ≠ permission for physicians to recommend pseudoscience
The real problem with Dr. Accad’s attitude, however, boils down to an issue that he did not really address: What is the obligation of a physician with respect to his or her recommendations to patients? All his misunderstanding of what advocates of science-based medicine actually argue and his apparent lack of understanding how biomedical science actually works are painful for me to read, but what’s very noticeable by its absence, other than in that Tweet, is upon what Dr. Accad thinks we should base our recommendations to patients on, if not the results of biomedical science? To be fair, Clay did try to bring up this issue of patient autonomy and the problematic ethics of not providing science-based information to patients and, worse, of offering mystical pseudoscience (like “energy medicine”) to patients as though it had scientific validity:
@michelaccad @drflanders @MayoClinic Not about judgement of patient. It’s about ethics of lying to patient.
— Clay Jones (@skepticpedi) June 8, 2015
@michelaccad @drflanders @MayoClinic @ScienceBasedMed Patient’s don’t have true autonomy if they aren’t given science-base information. — Clay Jones (@skepticpedi) June 8, 2015
Unlike Dr. Accad, I do have a big problem with physicians and hospitals, particularly academic medical centers, peddling unadulterated nonsense like reiki to patients because that’s not what patients come to us for. They come to us for our best evidence-based recommendations, and, as Clay and I have argued, without that they cannot have true autonomy to give informed consent because, lacking that and in particular if the information they are given is just plain wrong (or, as Clay put it, unadulterated nonsense), they lack the necessary information to make an informed decision. Embracing and recommending pseudoscience represents a profound betrayal of our patients.
Think about it this way. Medicine has historically only been as good as the science behind it. Back when Hippocrates and his followers were revolutionizing Greek medicine, they made a number of advances, not the least of which was overturning the widespread belief that disease was caused by the gods or supernatural forces, thus introducing the concept that disease could be treated once it was understood how it comes about. Unfortunately, Hippocrates and his followers could go no further because they didn’t have a testable scientific framework into which they could put their observations. Hippocrates posited humoral theory, in which disease was thought to be the result of imbalances in four “humors.” Traditional Chinese medicine had a similar concept. True, they substituted five elements for four humors, but the concept, namely that disease was the result of an “imbalance” of these humors or elements, was very similar.
For many hundreds of years afterward, medicine remained more or less stagnant. Bleeding, cupping, violent purging, endless botanical abstracts, heavy metals, and the like reigned as the preferred treatments, with all sorts of learned treatises written on the benefits and harms, the pros and cons, of each. Advances were few and far between until 1800s, when medicine began to embrace science. Indeed, Lewis Thomas in his book The Youngest Science, dated the beginning of the era of scientific therapeutics to the beginning of the antibiotic era in the 1930s, noting that, until the discovery of chemotherapy for infections, clinicians quite literally had almost nothing to offer the patient other than rest, good nursing care, and draining abscesses there were any. He had a point, although the earlier discovery of insulin could just as easily been used as a starting point.
It is true that there are many problems with science-based medicine, and we have written about quite a few of them right here on this very blog over the last seven and a half years. One of the worst is how the reimbursement model has changed to the point where doctors are disincentivized from spending more time with their patients, which makes it very difficult to actually deliver that human touch, to be the healer that Dr. Accad so pines for. However, to paraphrase Ben Goldacre, just because there are flaws in aircraft design does not mean that flying carpets work, and just because there are problems with science-based medicine does not imply that we should embrace or tolerate physicians recommending unadulterated nonsense like reiki to patients to fix these problems. Dr. Accad claims that “scientific medicine is truly nonsense,” but it’s clear that he does not know what scientific medicine actually is, which is why he constructs a straw man version of it that is nonsense. Contrary to what people like Dr. Accad and Dr. David Katz argue, it is not necessary to reject science and embrace pseudoscience in order to be a caring, empathetic holistic healer. Physicians can—and should—be such healers using science-based medicine.

   Yes, Jacqueline: EBM ought to be Synonymous with SBM
Kimball Atwood
Link
“Ridiculing RCTs and EBM”
Last week Val Jones posted a short piece on her BetterHealth blog in which she expressed her appreciation for a well-known spoof that had appeared in the British Medical Journal (BMJ) in 2003:
Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials
Dr. Val included the spoof’s abstract in her post linked above. The parachute article was intended to be humorous, and it was. It was a satire, of course. Its point was to call attention to excesses associated with the Evidence-Based Medicine (EBM) movement, especially the claim that in the absence of randomized, controlled trials (RCTs), it is not possible to comment upon the safety or efficacy of a treatment—other than to declare the treatment unproven.
A thoughtful blogger who goes by the pseudonym Laika Spoetnik took issue both with Val’s short post and with the parachute article itself, in a post entitled #NotSoFunny – Ridiculing RCTs and EBM.
Laika, whose real name is Jacqueline, identifies herself as a PhD biologist whose “work is split 75%-25% between two jobs: one as a clinical librarian in the Medical Library and one as a Trial Search Coordinator (TSC) for the Dutch Cochrane Centre.” In her post she recalled an experience that would make anyone’s blood boil:
I remember it well. As a young researcher I presented my findings in one of my first talks, at the end of which the chair killed my work with a remark that made the whole room of scientists laugh, but was really beside the point…
This was not my only encounter with scientists who try to win the debate by making fun of a theory, a finding or …people. But it is not only the witty scientist who is to *blame*, it is also the uncritical audience that just swallows it.
I have similar feelings with some journal articles or blog posts that try to ridicule EBM – or any other theory or approach. Funny, perhaps, but often misunderstood and misused by “the audience”.
Jacqueline had this to say about the parachute article:
I found the article only mildly amusing. It is so unrealistic, that it becomes absurd. Not that I don’t enjoy absurdities at times, but absurdities should not assume a life of their own. In this way it doesn’t evoke a true discussion, but only worsens the prejudice some people already have.
Jacqueline argued that two inaccurate prejudices about EBM are that it is “cookbook medicine” and that “RCTs are required for evidence.” Regarding the latter, she made reasonable arguments against the usefulness or ethics of RCTs for “prognostic questions,” “etiologic or harm questions,” or “diagnostic accuracy studies.” She continued:
But even in the case of interventions, we can settle for less than a RCT. Evidence is not present or not, but exists on a hierarchy. RCT’s (if well performed) are the most robust, but if not available we have to rely on “lower” evidence.
BMJ Clinical Evidence even made a list of clinical questions unlikely to be answered by RCT’s. In this case Clinical Evidence searches and includes the best appropriate form of evidence.
     1.  where there are good reasons to think the intervention is not likely to be beneficial or is likely to be harmful;
     2.  where the outcome is very rare (e.g. a 1/10000 fatal adverse reaction);
     3.  where the condition is very rare; [etc., for a total of 6 more categories]
In asserting her view of another inaccurate prejudice about EBM, Jacqueline took Dr. Val and Science-Based Medicine to task:
Informed health decisions should be based on good science rather than EBM (alone).
Dr. Val: “EBM has been an over-reliance on “methodolatry” - resulting in conclusions made without consideration of prior probability, laws of physics, or plain common sense. (….) Which is why Steve Novella and the Science Based Medicine team have proposed that our quest for reliable information (upon which to make informed health decisions) should be based on good science rather than EBM alone.”
Methodolatry is the profane worship of the randomized clinical trial as the only valid method of investigation. This is disproved in the previous sections.
The name “Science Based Medicine” suggests that it is opposed to “Evidence Based Medicine”. At their blog David Gorski explains: “We at SBM believe that medicine based on science is the best medicine and tirelessly promote science-based medicine through discussion of the role of science and medicine.”
While this may apply to a certain extent to quack[ery] or homeopathy (the focus of SBM) there are many examples of the opposite: that science or common sense led to interventions that were ineffective or even damaging, including:
     1.  protective effects of beta-carotene on lung cancer (opposite)
     2.  protective effects of vitamin E on cardiovascular disease (opposite) or certain cancers (neutral)
     3.  protective effects of the prone sleeping on Sudden Infant Death Syndrome (opposite)
     4.  Protective effects of corticosteroids in severe head trauma (opposite) etcetera.
As a matter of fact many side-effects are not foreseen and few in vitro or animal experiments have led to successful new treatments.
At the end it is most relevant to the patient that “it works” (and the benefits outweigh the harms).
Furthermore EBM is not -or should not be- without consideration of prior probability, laws of physics, or plain common sense. To me SBM and EBM are not mutually exclusive.
Jacqueline finished by quoting a few comments that had appeared on the BMJ website after the parachute article. Some of them (not all, I’m happy to report) revealed that their authors lacked a sense of humor. Another argued that “EBM is not RCTs.” Still others argued that RCTs are valuable for precisely the reason illustrated by Jacqueline’s examples listed above: that even some, seemingly safe and effective treatments—based on science or common sense or clinical experience—have eventually been shown, when subjected to RCTs, to behave otherwise. No one at SBM would argue the point.
Science-Based Medicine is Not Opposed to Evidence-Based Medicine
I am confident in asserting that we at SBM are in nearly complete agreement with Jacqueline regarding how EBM ought to be practiced. We are, I’m sure, also in agreement that many objections to EBM are specious. Among these, soundly criticized on this site, are special pleadings and bizarre post-modern arguments. The name “Science-Based Medicine” does not suggest that we are opposed to EBM. What it does suggest is that several of us consider EBM to be incomplete in its gathering of evidence, incomplete in ways that Jacqueline herself touched upon. I explained this in a series of posts at the inception of SBM in 2008,† and I discussed it further at TAM7 last summer. As such, Managing Editor David Gorski invited me to respond to Jacqueline’s article. I am happy to do so because, in addition to clarifying the issues for her, it is important to review the topic periodically: The problems with EBM haven’t gone away, but readers’ memories are finite.
Let me begin by asserting that everyone here agrees that large RCTs are the best tools for minimizing bias in trials of promising treatments, and that RCTs have repeatedly demonstrated their power to refute treatment claims based solely on physiology, animal studies, small human trials, clinical judgment, or whatever. I made the very point in my talk at TAM7, offering the Cardiac Arrhythmia Suppression Trial and the Women’s Health Initiative as examples. We also agree that there are some situations in which RCTs, whether for logistical, ethical, or other reasons, ought not to be used or would not yield useful information even if attempted. Parachutes are an example, but there are subtler ones, e.g., the efficacy of pandemic flu vaccines or whether the MMR vaccine causes autism. As we shall see, however, the list of exceptions offered by Jacqueline and BMJ Clinical Evidence is neither a formal part of EBM nor universally accepted by EBM practitioners.
To reiterate: The most important contribution of EBM has been to formally emphasize that even a high prior probability is not always sufficient to establish the usefulness of a treatment—parachutes being exceptions.
EBM’s Scientific Blind Spot
Now, however, we come to an important problem with EBM, a problem not merely of misinterpretations of its tenets (although such are common), but of the tenets themselves. Although a reasonably high prior probability may not be a sufficient basis for incorporating a treatment into general use, it is a necessary one. It is, moreover, a necessary basis for seriously considering such a treatment at all; that is, for both scientific and ethical reasons it is a prerequisite for performing a randomized, controlled human trial. Rather than explain these points here and now, I ask you, Dear Reader, to indulge me by following this link to a post in which I have already done so in some detail. I’ll wait here patiently.
……………….
Are you back? OK. Now you know that we at SBM are in total agreement with Jacqueline that EBM “should not be without consideration of prior probability, laws of physics, or plain common sense,” and that SBM and EBM should not only be mutually inclusive, they should be synonymous. You also know, however, that Jacqueline was mistaken to claim that EBM already conforms to those ideals. It does not, and its failure to do so is written right into its Levels of Evidence scheme—the exceptions that she offered, including those quoted from BMJ Clinical Evidence, notwithstanding. You know all of this because you’ve now seen several examples (there are many more) from that wellspring of EBM reviews, Jacqueline’s own Cochrane Collaboration. (There is another, more subtle reason for prior probability being overlooked in EBM literature, but it is an optional exercise for the purposes of today’s discussion).
EBM and Unintended Mischief
The problems caused by EBM’s scientific blind spot are not limited to the embarrassment of Cochrane reviews suggesting potential clinical value for inert treatments that have been definitively refuted by basic science, although that would be sufficient to argue for EBM reform. The Levels of Evidence scheme has resulted in dangerous or unpleasant treatments being wished upon human subjects in the form of RCTs, cohort studies, or case series even when existing clinical or scientific evidence should have been more than satisfactory to put such claims to rest. The Trial to Assess Chelation Therapy (TACT)—the largest, most expensive, and most unethical trial yet funded by the NCCAM—was originally justified by these words in an editorial in the American Heart Journal in 2000, co-authored by Gervasio Lamas, who would later become the TACT Principal Investigator:
The modern standard for accepting any therapy as effective requires that there be scientific evidence of safety and efficacy in a fair comparison of the new therapy to conventional care. Such evidence, when widely disseminated, leads to changes in clinical practice, ultimately benefitting patients. However, the absence of a clinical trial does not disprove potential efficacy, and a well-performed but too small “negative” trial may not have the power to exclude a small or moderate benefit of therapy. In other words, the absence of evidence of efficacy does not constitute evidence of absence of efficacy. These concepts constitute the crux of the lingering controversy over chelation therapy…
Such an argument, with its obvious appeal to the formal tenets of EBM, was made and accepted by the NIH in spite of overwhelming evidence against the safety and effectiveness of Na2EDTA chelation treatments for atherosclerotic vascular disease, including the several “small” disconfirming RCTs, comprising approximately 270 subjects, to which Dr. Lamas alluded. It was also accepted in spite of its violating both the Helsinki Declaration and the NIH’s own policy stipulating that preliminary RCTs should demonstrate efficacy prior to a Phase III trial being performed.
A 2006 Cochrane Review of Laetrile for cancer would, if its recommendations were realized, stand the rationale for RCTs on its head:
The most informative way to understand whether Laetrile is of any use in the treatment of cancer, is to review clinical trials and scientific publications. Unfortunately no studies were found that met the inclusion criteria for this review.
Authors’ conclusions
The claim that Laetrile has beneficial effects for cancer patients is not supported by data from controlled clinical trials. This systematic review has clearly identified the need for randomised or controlled clinical trials assessing the effectiveness of Laetrile or amygdalin for cancer treatment.
Why does this stand the rationale for RCTs on its head? A definitive case series led by the Mayo Clinic in the early 1980s had overwhelmingly demonstrated, to the satisfaction of all reasonable physicians and biomedical scientists, that not only were the therapeutic claims for Laetrile baseless, but that the substance is dangerous. The subjects did so poorly that there would have been no room for a meaningful advantage in outcome with active treatment compared to placebo or standard treatment—as we have recently seen in another trial of a quack cancer treatment. The Mayo case series “closed the book on Laetrile,” the most expensive health fraud in American history at the time, only to have it reopened more than 20 years later by well-meaning Cochrane reviewers who seemed oblivious of the point of an RCT.
A couple of years ago I was surprised to find that one of the authors of that review was Edzard Ernst, a high-powered academic who over the years has undergone a welcomed transition from cautious supporter to vocal critic of much “CAM” research and many “CAM” methods. He is now a valuable member of our new organization, the Institute for Science in Medicine, and we are very happy to have him. I believe that his belated conversion to healthy skepticism was due, in large part, to his allegiance to the formal tenets of EBM. I recommend a short debate published in 2003 in Dr. Ernst’s Focus on Alternative and Complementary Therapies (FACT), pitting Jacqueline’s countryman Cees Renckens against Dr. Ernst himself. Dr. Ernst responded to Dr. Renckens’s plea to apply science to “CAM” claims with this statement:
In the context of EBM, a priori plausibility has become less and less important. The aim of EBM is to establish whether a treatment works, not how it works or how plausible it is that it may work. The main tool for finding out is the RCT. It is obvious that the principles of EBM and those of a priori plausibility can, at times, clash, and they often clash spectacularly in the realm of CAM.
I’ve discussed that debate before on SBM, and I consider it exemplary of what is wrong with how EBM weighs the import of prior probability. Dr. Ernst, if you are reading this, I’d be interested to know whether your views have changed. I hope that you no longer believe that human subjects ought to be submitted to a randomized, controlled trial of Laetrile!
When RCTs Mislead
Finally, for the purposes of today’s discussion, let me reiterate another point that must be considered in the context of establishing, via the RCT, whether a treatment works: When RCTs are performed on ineffective treatments with low prior probabilities, they tend not to yield merely ‘negative’ findings, as most physicians steeped in EBM would presume; they tend, in the aggregate, to yield equivocal findings, which are then touted by advocates as evidence favoring such treatments, or at the very least favoring more trials—a position that even skeptical EBM practitioners have little choice but to accept, with no end in sight. Numerous such examples have been discussed on this website.
The first sentence that I ever posted on SBM, a quotation from homeopath David Reilly, was a perfect illustration of this misunderstanding:
Either homeopathy works or controlled trials don’t!
Dr. Reilly was correct, of course, but not in the way that he supposed. If there is anything that the history of parapsychology can teach the biomedical world, it is the point just made: human RCTs, as good as they are at minimizing bias or chance deviations from population parameters, cannot ever be expected to provide, by themselves, objective measures of truth. There is still ample room for erroneous conclusions. Without using broader knowledge (science) to guide our thinking, we will plunge headlong into a thicket of errors—exactly as happened in parapsychology for decades and is now being repeated by its offspring, “CAM” research.
Conclusion
These are the reasons that we call our blog “Science-Based Medicine.” It is not that we are opposed to EBM, nor is it that we believe EBM and SBM to be mutually exclusive. On the contrary: EBM is currently a subset of SBM, because EBM by itself is incomplete. We eagerly await the time that EBM considers all the evidence and will have finally earned its name. When that happens, the two terms will be interchangeable.

   Science-based medicine and improving patient safety and quality of care
David Gorski
Link
The last couple of weeks, I feel as though I may have been slumming a bit. After all, comparatively speaking it’s not that difficult to take on claims that homeopathy benefits fibromyalgia or Oprah Winfrey promoting faith healing quackery. Don’t get me wrong. Taking on such topics is important (otherwise I wouldn’t do it). For one thing, some quackery is so harmful and egregiously anti-science that it needs to be discussed. For another thing, they serve as examples of how even the most obvious quackery can seem plausible. All it takes are the cognitive quirks to which all humans are prone plus a bit of ignorance about what constitutes good scientific evidence to support the efficacy of a given therapy for a given condition.
So let’s move on to something a little more challenging.
f all the attacks on science-based medicine (SBM), one of the favorite attacks made by its opponents is the claim that SBM is dangerous, that it kills or harms far more people than it helps. An excellent example of this occurred when quackery promoter Joseph Mercola teamed up with fellow quackery promoter Gary Null to write a widely cited article entitled Death by Medicine. Using the famous Institute of Medicine article that estimated deaths from medical errors to be on the order of 50,000 to 100,000 per year, Mercola and Null wove a scary story meant to imply that conventional medicine does far more harm than good. Of course, as our very own Harriet Hall pointed out, they concentrated solely on the harm, which makes it difficult to determine whether the harms truly outweigh the benefits. As Peter Lipson puts it, such arguments are intentionally designed to take our fears and exaggerate them out of all perspective. The idea behind the fallacious arguments used by the likes of Mercola and Null is that, because “conventional” medicine has problems and needs to improve its safety record, the quackery they promote must be a viable alternative to SBM. Yes, that is basically what their arguments boil down to.
The fallacious manner in which advocates for quackery such as Joe Mercola, Mike Adams, and Gary Null use and abuse any shortcoming of SBM that they can find (and, when they can’t find any, make some up) notwithstanding, there is a problem in SBM. Indeed, over the last 10 years or so since the IOM report, reducing the toll due to medical errors has — finally — become an incredibly important issue in medicine. Indeed, I myself have become involved in a state-wide quality improvement initiative in breast cancer as our site’s project director. As a result, I’m being forced to learn more about the nitty-gritty of quality improvement than I had ever thought I would. Combine this with a study published just before the Thanksgiving holiday in the New England Journal of Medicine, and I’m learning that improving care is incredibly difficult. The issues involved are many and tend to involve systems rather than individuals, which is why the solutions often bump up against the individualistic culture to which physicians belong. Moreover, such efforts, like comparative effectiveness research (CER), tend to earn less prestige than scientific research because, like CER, quality improvement initiatives do not in general look for new information and scientific understanding but rather at how we apply what we already know.
Quality versus safety
Another thing that needs to be remembered is that there is a great deal of controversy over which outcomes should be measured as indicators of quality care, particularly now that third party payors are supporting “quality improvement” initiatives. However, if you look at what is being measured, more frequently they are far more measures of safety (or, more properly), lack of safety than measurements of quality. In other words, what is being measured with a view towards minimization are error rates: medication errors; wrong site surgery rates; failure to adhere to well-established science-based guidelines, such as perioperative antibiotics, adequate sterile technique, and the use of deep venous thrombosis prophylaxis in appropriate patients; and, in general, anything that puts patient safety at risk based on what we know already. We already know that these practices decrease complication rates. Administering appropriate perioperative antibiotics and not allowing the patient’s body temperature to fall too far during surgery, for example, both decrease postoperative infection rates, as has been documented extensively through numerous studies. Leaving urinary catheters in longer than they are absolutely necessary increases rates of urinary tract infections. Failing to get patients out of bed as soon as possible after routine abdominal surgery increases the rates of postoperative pneumonia and deep venous thromboses. And so it goes. Some of these result in outcomes that, if proper systemic measures are in place to prevent them, should be very, very close to zero:
The concept of never events sounds appealing. Certain things should never happen. There are some never events. The surgeon should never remove the wrong leg or take out the wrong kidney. They should never leave an instrument in your abdomen after surgery. Patients should never receive another patient’s medications.
Other outcomes besides these are coming to be defined as “never events,” outcomes such as catheter-associated infections or leaving surgical instruments or sponges behind in a patient’s abdomen. Indeed, Medicare is no longer paying for care associated with several of these so-called “never events.”
Quality, on the other hand, is a more difficult concept to define, as Dr. Robert Centor describes. I wouldn’t go so far as to call the definition of quality “nebulous,” as he does. To me, that characterization goes too far. Rather, I would simply describe quality as being an concept where there is a lot more disagreement over what, exactly, constitutes “quality” care. True, safe care is definitely part of quality. After all, how could anyone define unsafe care as being “high quality”? In other words, safety is a necessary but not sufficient requirement for medical care to be considered high quality. For example, Dr. Centor further elaborates:
I am happy to see safety rates reported, e.g. central line infection rates, PICC line clot rates and or infection rates. These differ greatly from performance measures like % of MI patients given beta blockers. True safety parameters are not influenced by patient factors. We should minimize line infections through system work. We should decrease urinary tract infection rates. My distinction is that we can impact the untoward consequences of the things we do TO patients. We can agree on these, and fixing these problems is JOB #1.
After that, defining quality gets more difficult. Does it mean following more global science-based guidelines, such as the National Comprehensive Cancer Network‘s (NCCN) or the American Cancer Society‘s guidelines for the care of various cancers. These are different from simple quality measures in that they represent treatment algorithms designed more to optimize the overall care of cancer rather than relatively simple practices designed to protect patient safety. For example, one of the NCCN’s treatment guidelines for breast cancer requires that radiation therapy be offered to women who have undergone breast-conserving therapy within a certain time frame and that women with certain types and sizes of tumor be offered adjuvant chemotherapy. For purposes of this post, I will be sticking with safety measurements for the most part, because such measures are much more concrete and less contentious.
Checklists: Simple but overlooked
One of the most important points about quality improvement initiatives is that in general they emphasize doing what SBM already tells us to be the best practices. Examples include practices as obvious as giving perioperative antibiotics at the right time before operations and for the right operations. (For example, “clean” operations generally don’t need perioperative antibiotics.) One way to look at this is to give the right drug for the right type of operation at the right time before the operation until the right time after the operation. For most operations requiring perioperative antibiotics, the correct time to give the dose is a sufficient length of time to achieve therapeutic blood levels before the incision is made. If you administer the antibiotics after the incision has been made, you might as well not give them at all; at that point they don’t work to decrease the rate of infection.
One area that has received a fair amount of attention lately is the issue of infections and sepsis due to indwelling central venous catheters. Central venous catheters are catheters inserted directly into one of the large veins that drain to the heart, such as the internal jugular or subclavian vein. There, they frequently remain for days, sometimes for weeks. Because they are foreign objects that penetrate the skin and sit in large veins for (sometimes) long periods of time, central venous catheters can be the nidus for infection, and catheter-associated sepsis has traditionally been a difficult problem. Over the years, various practices have been advocated for reducing the rate of line-associated sepsis, including changing the line every three days and various other practices. As Atul Gawande pointed out in a famous New York Times editorial three years ago, however, it turns out that the most effective method for reducing catheter-associated sepsis:
A year ago, researchers at Johns Hopkins University published the results of a program that instituted in nearly every intensive care unit in Michigan a simple five-step checklist designed to prevent certain hospital infections. It reminds doctors to make sure, for example, that before putting large intravenous lines into patients, they actually wash their hands and don a sterile gown and gloves.
The results were stunning. Within three months, the rate of bloodstream infections from these I.V. lines fell by two-thirds. The average I.C.U. cut its infection rate from 4 percent to zero. Over 18 months, the program saved more than 1,500 lives and nearly $200 million.
Amazing. Just making sure to do an operating room-style scrub and then donning sterile gowns and gloves has an amazing effect on infection rates. Who knew? Actually, we all did–or, at least we should have.The problem was that surgeons inserting these catheters weren’t doing what they should have been doing. The reasons for this were probably multiple: Lack of time, the “it’s the way I was trained” excuse, and an inflated sense of how well each doctors were doing sterile technique, for example. As I put it at the time when I read Dr. Gawande’s article and the associated study, this is remedial Infection Control 101 for dummies. (Actually, I used a different word than “dummies.”)
This is the sort of thing that is meant by “safety,” namely doing very simple things that SBM tells us that we should be doing anyway. (The original study can be found here, for those interested.)
Increasingly, medicine is taking a page from the airline industry and instituting these checklists. Another study in which a checklist appears to have had salutary effects on patient safety was published by Dr. Gawande last year in the NEJM. The checklist tested involved some fairly basic things. It was divided into three sections. The “sign-in” section had typical elements, such as verifying patient identity; making sure that surgical site is properly marked with a marking pen; making everyone aware of any patient allergies; checking the pulse oximeter; and determining if there is a risk of aspiration or significant blood loss. The preop “time out” involved such things as once again verifying the patient identity and surgical site; making sure that any preop antibiotics have been given less than 60 minutes prior to skin incision; confirming that all appropriate imaging results were available, correct, and show what they are reported to have shown; and reviewing anticipated critical events. Finally, the “sign out” phase involved the typical surgical necessity of making sure that the sponge and instrument counts were correct before waking the patient up; checking any specimens to make sure they were labeled correctly; and reviewing aloud concerns about any issues that might interfere with the recovery of the patient.
These measures were implemented at eight hospitals scattered throughout the world, both in developed countries and Third World countries. Sites included London, Seattle, Toronto, Tanzania, India, Jordan, the Philippines, and New Zealand. These hospitals had varying safety measures in place before the study, and none of them were as extensive as the checklist introduced as part of the study. The results are summarized in the following table (click to enlarge):
As you can see, the results were quite striking. But was it the introduction of checklists that resulted in this improvement? That’s difficult to say definitively from just this study. Certainly it is possible, but exactly how is harder to ascertain. One possibility is that the introduction of such checklists leads to a systemic improvement in how care is delivered driven by adherence to the principles embodied in the checklist. As the authors speculate in the discussion:
Whereas the evidence of improvement in surgical outcomes is substantial and robust, the exact mechanism of improvement is less clear and most likely multifactorial. Use of the checklist involved both changes in systems and changes in the behavior of individual surgical teams. To implement the checklist, all sites had to introduce a formal pause in care during surgery for preoperative team introductions and briefings and postoperative debriefings, team practices that have previously been shown to be associated with improved safety processes and attitudes and with a rate of complications and death reduced by as much as 80%. The philosophy of ensuring the correct identity of the patient and site through preoperative site marking, oral confirmation in the operating room, and other measures proved to be new to most of the study hospitals.
One interesting aspect of this initiative (interesting to me, at least) is the level of hostility to the results of the catheter study and this study that I encountered after they first came out. Doctors tend to resist checklists. I don’t know if this is anything unique to physicians, but given that airline pilots have used preflight checklists for years I’ve always had a hard time understanding the visceral reaction that such lists seem to cause in physicians. It’s as though too many of us consider ourselves so superior to our fellow human beings that the utility of such lists do not apply to us, we never forget to do anything, and never need reminding to do what we should be doing in the first place. To some extent, I can understand this reaction in that I used to make fun of what until about three years ago I used to disparage as the “mindless ritual” of my having to mark which breast I was going to operate on and to do a “time out” to verify it with the entire surgical staff in my operating room. I even used to say that once something is made into a rigid policy or checklist, it’s an excuse to stop thinking and mindlessly go through the motions. To some extent, I still think it’s important to guard against that tendency, but I’m far more supportive of checklists than I used to be.
In any case, the resistance of physicians to these sorts of checklists and various measurements that I mentioned at the beginning of this post appears to be due at least partially to the individualistic culture of physicians, but it should also be pointed out that malpractice concerns also appear to play a role. Indeed, sometimes this focus on the skills of the individual practitioner leads groups of physicians to resist even evidence-based medicine. In order to figure out how to prevent events that threaten patient safety, it is necessary to have a free and frank discussion of how such events occur, just as airlines do after crashes and near-misses.
But are checklists and safety measures having an effect?
While I’m sure most, if not all of us, can agree that this relatively newfound emphasis on improving patient safety and quality of care through guidelines, checklists, and other methods designed to ensure that, at the very least, the simple things that we know to be effective at improving patient outcomes are in fact being done, is long overdue, we still have to evaluate whether or not these measures actually work. In other words, are the rates of medical errors decreasing? Over the last couple of months, there have been two studies that are depressing in that they imply that the answer to that question might well be, “No.”
First, let’s take a look at one specific “never event,” namely wrong site or wrong patient surgeries. A recent study from the University of Colorado published in October in the Archives of Surgery, for example, found that wrong site surgeries, although rare, are more common than previous estimates, accounting for approximately 0.5% of all medical mistakes noted in the study. In a database of 27,370 physician self-reported adverse occurrences during a time period spanning January 1, 2002, to June 1, 2008, investigators identified 25 wrong patient and 107 wrong site surgeries. Significant harm was reportedly inflicted in 5 wrong-patient procedures (20.0%) and 38 wrong-site procedures (35.5%), and one patient died secondary to a wrong-site procedure (0.9%). Root cause analysis showed that most wrong-patient procedures were due to errors in diagnosis (56.0%) and errors in communication (100%), whereas wrong-site occurrences were related to errors in judgment (85.0%) and the lack of performing a “time-out” (72.0%). It should be pointed out that specialties were involved in the cause of wrong-patient procedures and contributed equally with surgical disciplines to adverse outcome related to wrong-site occurrences.
It should be also pointed out that there isn’t much in the way of convincing evidence that the Universal Protocol is currently helping.
Finally, as I mentioned above, right there in the New England Journal of Medicine last week was a study that specifically looked at trends over time for medical errors that caused patient injury. In brief, investigators at Harvard and Stanford Universities retrospectively examined the rate of medical errors at ten North Carolina hospitals from 2002 to 2007, chosen for this reason, as described in the introduction:
We chose North Carolina as a site that was likely to have improvement, since it had shown a high level of engagement in efforts to improve patient safety, including a 96% rate of hospital enrollment in a previous national improvement campaign, as compared with an average rate of 78% in other states, and extensive participation in statewide safety training programs and improvement collaboratives.
So what did these investigators find? Let’s just put it this way: This is one case where a picture is worth the proverbial thousand words:
This is Figure 2 from the study. As you can see, there is precious little, if any, improvement in the incidence of harm due to preventable medical errors, at least over the period studied from 2002 to 2007.
Where now?
One might conclude from the previous two studies that I presented that thus far all the safety/quality improvement modalities that have been implemented over the decade since the IOM report have been useless. However, although I do believe that boosters of checklists and various other safety improvement measures probably oversell their ability to make a dent in the rate of medical errors in American hospitals, I also don’t subscribe to the nihilistic view that none of this does any good. For one thing, it is very clear that in certain areas, carefully targeted checklists and other interventions can result in decreases in errors. However, medical errors span many specialties and many institutions, which means that, although, for example, central line checklists might very well greatly decrease the incidence of catheter-associated sepsis, such errors might represent a small enough percentage of overall medical error-associated harm that even huge decreases in such a complication don’t yet show up above the random noise and year-to-year fluctuations in the incidence of harm due to medical errors. Wrong-site surgeries, for instance, although catastrophic “never events,” are still, thankfully, rare; eliminating them, as important as that is, would not be expected to decrease overall rates of harm from medical errors by much. Unfortunately, we don’t appear even to be making much headway in this area.
Alternatively, one can argue, as the authors of these last two studies do, that we haven’t yet gone far enough in our safety interventions. For example, measures designed to prevent wrong site and wrong patient surgery are only implemented in the operating rooms:
But these protocols “are not sufficient,” Stahel says. They only apply to the operating room, he says, and nearly one-third of the botched procedures in the study took place in doctor’s offices. Moreover, the study showed that many operating-room mistakes start out in biopsy labs or during the imaging and diagnosis process.
“A lot of wrong-side, wrong-patient errors occur outside of the operating room,” Stahel says. “We should have time-outs for labeling for samples. If the lab mixes up the sample, the consequences may be worse than erroneously cutting off the wrong leg. I think we should extrapolate time-outs to internal medicine [and] laboratories.”
In the discussion of the NEJM study, the authors comment:
Our findings validate concern raised by patient-safety experts in the United States1 and Europe1 that harm resulting from medical care remains very common. Though disappointing, the absence of apparent improvement is not entirely surprising. Despite substantial resource allocation and efforts to draw attention to the patient-safety epidemic on the part of government agencies, health care regulators, and private organizations, the penetration of evidence-based safety practices has been quite modest. For example, only 1.5% of hospitals in the United States have implemented a comprehensive system of electronic medical records, and only 9.1% have even basic electronic record keeping in place; only 17% have computerized provider order entry.13 Physicians-in-training and nurses alike routinely work hours in excess of those proven to be safe. Compliance with even simple interventions such as hand washing is poor in many centers.
A reliable measurement strategy is required to determine whether efforts to enhance safety are resulting in overall improvements in care, either locally or more broadly. Most medical centers continue to depend on voluntary reporting to track institutional safety, despite repeated studies showing the inadequacy of such reporting. The patient-safety indicators of the AHRQ are susceptible to variations in coding practices, and many of the measures have limited sensitivity and specificity. Recent studies have shown that the trigger tool has very high specificity, high reliability, and higher sensitivity than other methods. Manual use of the trigger tool is labor-intensive, but as electronic medical records become more widespread, automating trigger detection could substantially decrease the time required to use this surveillance tool.
In other words, our ability to detect medical errors in retrospective studies is maddeningly inadequate, there is no standardization in how errors are reported or even mandatory reporting of such errors, and the efforts we have implemented have been largely confined to hospitals, when most patient receive care in the outpatient setting, including increasingly invasive procedures. Clearly, much more needs to be done. Checklists, such as the Universal Protocol, can only be part of the solution.
As disappointing as this is, I do find reason for optimism. Increasingly, regulatory agencies and third party payors are requiring the implementation of measures to improve patient safety. The problem is that, with the exception of some very obvious interventions, such as requiring hand-washing and sterile gowning before inserting central lines, some of these interventions have not been as well studied as they should be. For instance, it is widely claimed that the adoption of electronic medical records will allow for greater patient safety through the automation of checklists and various other safety checks. It is not yet clear that this will be true as EMRs are more widely adopted.
It should not be surprising that decreasing the rate of medical errors and the harm they cause does not lend itself to easy answers. The reasons for medical errors are many and multifarious, whether you accept the IOM’s estimates for the number of deaths caused by them or consider it flawed and likely an overestimate or an estimate without the context of what the expected death rate in hospitals would be in the absence of medical errors. Solving the problem and minimizing medical errors that result in harm will similarly not yield to a single strategy; our attacks on this problem will have to be as multi-pronged as the problem.

HHS is weaponizing evidence-based medicine to falsely portray vaccines as unsafe
Last week, HHS announced that all “new vaccines will undergo safety testing in placebo-controlled trials.” Sounds good? Not so fast. This is really a deceptive weaponization of evidence-based medicine to undermine confidence in vaccines and eliminate at least some of them.
David Gorski on May 5, 2025
Link
Having a man who has for the last two decades arguably been America’s most famous and intellectual antivax activist in charge of the Department of Health and Human Services (HHS) has been exhausting. Every week, I say that I’m not going to write about whatever new horror Robert F. Kennedy Jr. is inflicting on US federal health, public health, and biomedical research, but then every week he comes up with something new that, to those of us paying attention, is not really new at all. Last week, it was this announcement showing up in all sorts of news stories, for example, The Washington Post:
Health and Human Services Secretary Robert F. Kennedy Jr. intends to shift the way vaccines are tested, a move that the agency said will increase transparency but that medical experts fear could limit access to vaccines and undermine the public’s trust in immunization depending on its implementation.
The potential change outlined in a statement says all new vaccines will be required to undergo placebo testing, a procedure in which some people receive the vaccine and others receive an inert substance — such as a saline shot — before the results are compared.
“All new vaccines will undergo safety testing in placebo-controlled trials prior to licensure — a radical departure from past practices,” an HHS spokesperson told The Washington Post in response to questions about Kennedy’s comments on the measles vaccines and general vaccine policy.
As soon as I started reading these stories and seeing them pop up on TV in news reports, I realized that this was simply the implementation of something that antivaxxers and COVID-19 contrarians had been doing all during the pandemic, riffing on an antivax trope that long predated the pandemic. Indeed, less than a year and a half ago, I wrote a post about it entitled 2023: The year that the evidence-based medicine (EBM) paradigm was weaponized against vaccines and public health. Sadly, I never thought I’d be doing an update to this post in which I felt the need to explain that this weaponization of the EBM paradigm is now federal vaccine policy—and, no doubt, soon to be policy regarding all public health—but here we are.
Think I’m overreacting? Here’s the “tell” from the HHS statement:
In recent weeks, the Trump administration has injected uncertainty into the annual process for approving an updated coronavirus shot traditionally offered in the fall, indicating there may need to be more data.
“Except for the COVID vaccine, none of the vaccines on the CDC’s childhood recommended schedule was tested against an inert placebo, meaning we know very little about the actual risk profiles of these products,” HHS said in its statement.
This is a straight-up antivax trope, one that I’ve written about a number of times, specifically the claim that none of the vaccines on the CDC’s childhood recommended vaccine schedule was tested against an “inert placebo.” Usually the antivax version of this specifies a saline placebo as The Only One True Inert Placebo permitted, but, whatever the term, the trope remains. It’s a trope long repeated by RFK Jr. and Aaron Siri, an antivax lawyer with whom RFK Jr. has frequently worked, and therefore one that I’ve written about before, specifically how the trope is based on a deceptive half-truth based on on a misuse of EBM that I like to call methodolatry.
Before I discuss why that is, I also have to note that it’s a straight up lie to claim that “none of the vaccines on the CDC’s childhood recommended schedule was tested against an inert placebo” (emphasis mine). The HHS statement itself even concedes that the COVID-19 vaccine, which is currently—although probably not for much longer—on the CDC’s recommended childhood vaccination schedule was tested against placebo. That actually surprised me, given the RFK Jr.’s propensity to lie about vaccines. In any case, it’s easy to see that the varicella vaccine, for example, was tested against saline placebo.
I’ll start by defining “methodolatry,” after which I’ll explain why it is, depending on your point of view, either a weakness of the EBM paradigm that is baked in or a fundamentalist version of EBM that ignores medical ethics in favor of supposed scientific rigor. Then I’ll discuss why this new HHS diktat, although represented as “gold standard science”—a term RFK Jr. and his minions love to throw around and something they claim to be “restoring” to federal vaccine policy—is, in actuality, a strategy to undermine trust in vaccines by portraying vaccines on the current CDC childhood schedule as inadequately tested and potentially unsafe. It is also a strategy that, depending on how “new vaccine” is defined, could prevent the approval of updated COVID-19 and flu vaccines and even provide a seeming “gold standard science” rationale for delicensing some currently licensed vaccines, all while falsely claiming the scientific high ground in doing so.
Think of it this way. The reality-challenged insinuation that the current childhood vaccine schedule is unsafe because its constituents haven’t been tested in placebo controlled RCTs notwithstanding, placebo-controlled RCTs demonstrating efficacy and safety have always been required for new vaccines against diseases that currently have no licensed vaccine.
Methodolatry: EBM fundamentalism that ignores ethics and practicality
Way, way back in the day in 2009, I came across the term “methodolatry” in a blog post by a senior epidemiologist who blogged under the ‘nym “revere,” who during the H1N1 influenza pandemic, called out a hagiography published in The Atlantic of Dr. Thomas Jefferson, an EBM guru at the Cochrane Collaborative known for having cast doubt on the efficacy and safety of flu vaccines, setting up his use of the term thusly:
Brownlee and Lenzer rely upon (and romanticize as a martyr and truth-teller) Dr. Thomas Jefferson, someone who is fast establishing himself as an “Evidence Based Medicine” (EBM) crank who who courts notoriety by being a contrarian. The kind of EBM practiced by the likes of Jefferson and some other randomized trials zealots is far from the judicious weighing of the evidence envisaged by its early proponents.
Sound familiar? Let’s just say that EBM fundamentalist zealots like Drs. Vinay Prasad, John Mandrola, and Adam Cifu are nothing new. Indeed, they are simply the latest generation of methodolatrists, “intellectual”—if you can call it that—descendants of, for example, Tom Jefferson and Peter Gøtzsche, who used it to cast doubt on the safety of Gardasil. (Never mind that the existence of long term studies like this one that did compare HPV vaccines to saline placebo controls rather undermines this particular antivaccine talking point.) The difference, of course, is that that Prasad and colleagues have way less experience in actually doing EBM. But what is methodolatry? Heed the words of revere, nearly 16 years ago:
There is nothing judicious about Jefferson, whose problem was described by one of my colleagues as “methodolatry,” the profane worship of the randomized clinical trial as the only valid method of investigation.
The very first comment after revere’s post describes the consequences of methodolatry, or what I sometimes also call EBM fundamentalism:
Even if in his case the agenda might be “That’s sufficient evidence?”, which if thoroughly applied would declare many treatments to be not effective and ultimately—if you follow the logic of EBM—not approved or allowed?
My specific point is that we have here a concrete example of the misuse of EBM that is having real world effects, perhaps to the point that one or more people will die from novel H1N1 because this article convinced them the benefit of getting inoculated is so low it doesn’t outweigh the perceived risks.
Again, does this sound familiar? It’s exactly what has been done by COVID-19 contrarians deceptively claiming the scientific high ground by demanding randomized controlled clinical trials (RCTs) on everything from mask mandates to other nonpharmaceutical interventions, while almost never actually proposing how one would do such a trial in the middle of a pandemic. As our very own Dr. Jonathan Howard has frequently pointed out, it’s all performative, designed to cast doubt on existing evidence and demand a standard of evidence that in the real world is often impossible to attain for whatever interventions they don’t like, be they COVID-19 boosters, masks, or whatever.
As I wrote at the time, you could take a number of articles citing the lack of RCTs about various non-pharmaceutical interventions for COVID-19 or for COVID-19 boosters or for a number of issues that have come up during the pandemic and change the words “novel H1N1” to “novel coronavirus” or “COVID-19,” and revere’s observations are just as valid today as they were in 2009. The only difference today is that far more EBM gurus seem to be promoting the same simplistic and misleading message that Tom Jefferson was back then; worse, the same idea is metastasizing from COVID-19 to the childhood vaccination schedule. If you doubt me, must look at how Vinay Prasad, eager to suck up to RFK Jr., suggested that the childhood vaccination schedule should undergo a “a multi-arm, factorial RCT…before public distrust gets too out of control,” without, as I pointed out, ever actually saying how such a massive trial could be done ethically. (Hint: It can’t, for reasons that I will explain.) Indeed, Dr. Prasad is prone to proposing RCTs for questions that are not even medical, such as testing various methods of deciding which grant applications are funded by the NIH.
Why is methodolatry so deceptively appealing? In brief, the way it is being used now by HHS under RFK Jr. depends on a very simplistic version of EBM as promoted by EBM/RCT zealots and understood by those knowledgeable enough to be aware of the basic tenets of EBM but who do not understand EBM sufficiently well to know the caveats; i.e., when we have to do with “lesser” evidence.
At this point, it’s useful to briefly explain the EBM paradigm that puts RCTs near the very top of the EBM pyramid—or hierarchy—of evidence, with the only evidence ranking higher being systematic reviews and meta-analyses of multiple RCTs. First, though, keep in mind these two bedrock principles of bioethics, the first relevant to all human subjects research including RCTs, the second being more relevant to RCTs:
     1.  No subject should receive anything less than current standard-of-care medicine, be it the intervention being tested or any other medical intervention that they might undergo during the trial.
     2.  There must be genuine clinical equipoise between all groups, including control groups, in any RCT. Clinical “equipoise” means that there must be genuine scientific uncertainty regarding whether the experimental intervention is truly better than what the control group receives. Obviously, physicians don’t carry out clinical trials if they don’t think that the intervention being tested is likely to be at least as good as the control group, and one can argue about what constitutes an adequate level of “uncertainty” to justify an RCT, but clinical equipoise is one bedrock principle that determines if an RCT can be ethical. In other words, you can’t ethically randomize human subjects to an experimental group that you know to be an inferior treatment.
Keep these principles in mind as I discuss how the EBM paradigm can become methodolatry. I will then describe how the HHS diktat could easily be methodolatry, depending upon how it is operationalized.
The EBM paradigm vs. methodolatry
The EBM paradigm is often described in terms of the “evidence pyramid,” in which types of scientific evidence applied to medicine are ranked thusly:
EBM pyramid of evidence. Note that anything that is not an RCT is automatically considered “lower quality” evidence and that the highest form of evidence of all is meta-analyses and systematic reviews of RCTs.
Randomized controlled clinical trials are just that: clinical trials in which subjects are randomized to receive either the intervention or a control (such as a saline placebo) and then followed to determine which group has better outcomes, which are prespecified in the clinical trial protocol. The reason for randomization is to ensure that the two groups being compared resemble each other as closely as possible in characteristics relevant to the outcomes being tested. For example, if you’re testing a drug to treat hypertension, you would want the groups to be matched as closely as possible for, among other characteristics, age, race, sex, severity of hypertension, and relevant risk factors for poor outcomes. Ideally, these RCTs are then double-blinded, so that neither the subjects nor the doctors or medical personnel administering the drugs and assessing outcomes know which group any given subject is in. Moreover, such clinical trials have strict inclusion and exclusion criteria, which ensure that those being treated actually have the disease, do not belong to a group that might be harmed by the drug, and are subjects who are likely to benefit if the drug does have efficacy; i.e., does work.
As you can see in the image above, in the EBM paradigm RCTs are ranked near the top of the pyramid for strength of evidence. Actually, as far as original clinical studies go, they are at the top. It is true that systematic reviews and meta-analyses of RCTs are ranked higher (i.e., as more rigorous), but remember that these are syntheses of existing RCTs, not original experimental clinical studies in which one intervention is tested against placebo or another intervention. Moreover, it is also true that, for certain questions, RCTs truly are the gold standard, as an RCT is the design that best minimizes bias, thanks to randomization and careful matching of the experimental versus the control group, plus blinding, which when done properly minimizes bias due to observation.
Let’s look at a concrete example that I like to cite, the strict EBM paradigm as applied to whether the MMR vaccine causes autism. (RFK Jr. has been saying for many years now that he believes it does.) Regular readers of SBM likely already know that in reality the evidence base of studies examining the question of whether vaccines cause autism is very large and has failed to find even hint of a whiff of an association between MMR (or MMRV) and autism. Yet, because the bulk of these studies consists of epidemiology, including one that encompassed 537,303 children representing 2,129,864 person-years of study, the certainty of the evidence behind the conclusion of the relevant Cochrane review that the MMR vaccine does not cause autism can never be more than “moderate” because by the very definition of the EBM paradigm and its ranking of sources of evidence, epidemiological studies always rank below RCTs. Note that it will never be possible to do an RCT to answer the question of whether vaccines like MMR increase the risk of autism because such a trial would be inherently unethical, not to mention also very impractical, as defenders of science-based vaccine requirements were explaining 15 years ago.
I discussed the nitty gritty of EBM levels of evidence the last time around, including some newer definitions and acknowledgment that certain questions can’t be answered by RCTs, if only for ethical reasons. I’ve also discussed how the EBM paradigm can “inspire” those who take it too far to want to argue that RCTs are the only valid method of investigation, even for questions for which they are ill-suited. I also note that EBM fundamentalists frequently ignore the precept that “if RCTs are impractical or unethical,” then cohort and case control studies must suffice. While it’s always difficult to tell whether EBM fundamentalists are true believers or are intentionally misusing the EBM paradigm of RCTs über alles, in the case of RFK Jr. I am willing to say that it is almost certainly intentional misuse of the paradigm. Let me explain through the example of the childhood vaccination schedule and the longstanding antivax half-truth that childhood vaccines have never been tested in proper RCTs.
RFK Jr: Weaponizing EBM fundamentalism against the childhood vaccination schedule
The idea that the vaccines in the childhood vaccination schedule have never been tested in “proper” RCTs for safety and efficacy is an old antivax trope, long championed by Aaron Siri, a shyster—IMHO, of course!—who’s long worked with antivax activists RFK Jr. and Del Bigtree and his antivax organization ICAN harassing various governments and companies with—again, IMHO—frivolous lawsuits designed to cast doubt on the safety and efficacy of vaccines. It was the central subject of an antivax book, Turtles All the Way Down, whose title appears to be an analogy to the actual situation with the childhood vaccination schedule, as you will see, particularly if you look at the book’s cover art:
Notice the turtles piled up on top of turtles going down ad infinitum. Besides being an allusion to Hindu mythology, in this case it’s referring to vaccines in the CDC childhood vaccine schedule and RCTs, as you will see.
The central premise of Siri and RFK Jr.’s claim that most childhood vaccines have not been tested against a saline placebo—rephrased in the HHS statement as “inert placebo”—has also been misrepresented as a “placebo pyramid scheme”:
Aaron Siri Exposes the 'Placebo Pyramid Scheme' Used to License Childhood Vaccines

"What Dr. Offit wrote is 'all vaccines are tested in placebo-controlled trials before licensure'…The problem is that is not true…He is dead wrong in making that claim…As an example the… pic.twitter.com/W2T5xtLpp9
— Chief Nerd (@TheChiefNerd) August 12, 2023
Or, as Dr. Vincent Iannelli at Vaxopedia represented it
It is, of course, true that many of the current vaccines in the childhood vaccine schedule were never tested an RCT vs. saline placebo, but there is a very good reason for that. Go back to the first ethical principle of human subjects research that I listed above. I’ll repeat it again for emphasis:
No subject should receive anything less than current standard-of-care medicine, be it the intervention being tested or any other medical intervention that they might undergo during the trial.
Why is this principal of clinical trial ethics so important regarding the childhood vaccination schedule? Simple. Consider first the case of a disease for which there is no existing FDA-licensed safe and effective vaccine; i.e., a first generation vaccine. To test an experimental first-generation vaccine, it is ethical to do an RCT, because there currently exists no vaccine that is licensed and standard-of-care, and that’s exactly what is required of such a new vaccine. In fact, that is what has always been required for new first generation vaccines. It’s the reason why, to be approved for use in children, the COVID-19 vaccines were required to undergo an RCT in children. Even the HHS statement conceded that COVID-19 vaccines had undergone placebo-controlled RCTs in children before being licensed.
Now let’s consider the case of a new vaccine for a disease for which there already exists a licensed vaccine that is safe and effective and considered by pediatricians to be standard-of-care, for example, vaccines against measles; i.e., a second, third, or even fourth generation vaccine. (I’ll call these second generation vaccines, for brevity’s sake.) Alternatively, consider the development of multivalent vaccines; i.e., multiple existing vaccines against different diseases combined into one shot for convenience of administration, such as measles-mumps-rubella (MMR) or measles-mumps-rubella-varicella (MMRV). In the case multivalent vaccines combining approved vaccines and of second generation vaccines, there already exists a vaccine licensed as safe and effective against an specific disease (or safe and effective vaccines against specific diseases) that are considered standard of care by pediatricians and other relevant medical specialties.
In these cases, other than rare cases, doing an RCT with a saline or “inert” placebo would be completely unethical because it would require randomizing a number of subjects to be in a group that receives less than the standard of care with respect to vaccines and who would therefore be intentionally left vulnerable to the diseases prevented by both the old vaccine and the new vaccine being tested. In the case of, for example, a new vaccine against a disease, the only ethical way to carry out an RCT of a second generation or of a new multivalent vaccine would be to test the new vaccine against the existing vaccine for that disease in order to determine that the new vaccine is at least as effective and safe as the old vaccine. This is called a non-inferiority trial. Even if a non-inferiority trial doesn’t show that the new vaccine is better than the old at preventing disease, there might still be reasons to favor the new vaccine even if it is only “as good as” the older vaccine. Reasons for favoring the new vaccine might include, for example, that it’s cheaper, easier to dose, or has a more tolerable side effect profile. Of course, ideally, we would prefer to be able to show that the new drug is better than the existing standard-of-care, but this is often very difficult to do, which is why noninferiority trials are more common.
In addition, remember the second ethical principle I mentioned. Regardless of the scientific rigor of the trial, randomization requires clinical equipoise; i.e., the existence of genuine uncertainty based on existing science and evidence about which group, control or experimental, will do better. Clinical equipoise is not just a requirement, but absolute requirement for randomization of clinical trial subjects to be considered ethical. Note that clinical equipoise does not mean that there is no evidence that a treatment works or is better than existing treatments. After all, the reason we do clinical trials is to find drugs that work where there currently are none and to find drugs that work better than current therapy, and there has to be compelling preclinical evidence from laboratory and animal studies, as well as preliminary early stage clinical trials, to justify testing a drug or vaccine in an RCT. There also has to be genuine scientific uncertainty, even if, as is usually the case, investigators think that it is more likely than not that the experimental group will do better than the control group. Absent that legitimate uncertainty, an RCT to test an intervention is unethical, as is the case in randomizing a child to be left unprotected against measles to test a new measles vaccine or formulation of vaccines that includes the measles vaccine.
I note that these two principles are general principles governing all clinical trial ethics as well. In my area of specialty, for instance, oncology, we don’t test new chemotherapeutic drugs against placebo if there already exist effective chemotherapy regimens with a side effect profile deemed tolerable. Instead, we either add the new drug to the existing chemotherapy regimen and test it against the existing chemotherapy regimen without the new drug, or, in the case where the existing chemotherapy regimen is only marginally effective, it can sometimes be ethical to test the new chemotherapy against the existing chemotherapy regimen. The principle to be remembered is that in no case should any human subject enrolled in the clinical trial ever receive anything less than the existing science-based standard-of-care. To do otherwise would be unethical. Indeed, this is codified in a multinational set of guidelines to which the FDA has adhered. I cited above and that could be fit into the FDA regulation that allows an active treatment concurrent control:
The use of a placebo control group does not imply that the control group is untreated. In many placebo-controlled trials, the new treatment and placebo are each added to a common standard therapy (so called add-on studies, see section 2.1.5.2.1).
Coming back to the childhood vaccine schedule, the true situation is that, in any cases for which a current vaccine in the schedule wasn’t tested against placebo, if you go back far enough you will inevitably find that the original vaccine against that disease was tested in a placebo-controlled RCT. As friend of the blog Skeptical Raptor put it:
So, let’s be clear here. Any new vaccines that replace older vaccines will probably not use a true placebo-controlled group. The new vaccine will almost always be compared against the older version of the vaccine, but there may be a way around that; I don’t see it.
Nor do I. As Dr. Stanley Plotkin puts it:
Stanley Plotkin, a pioneer in the field who developed the rubella vaccine, said that when scientists test vaccines against a new disease, they typically look for evidence that the vaccinated individuals do not get a disease, compared with those who received a placebo. This is how the coronavirus vaccines were tested, in 30,000-person trials in which half of the participants received saline shots.
But when a disease is already well understood, scientists can look for evidence that vaccines induce a biological response that has been scientifically shown to protect against the disease — what scientists call “a correlate of protection.”
In the case of diseases that cause serious illness and can even be fatal, if there are existing interventions, the use of placebos is often not considered ethical.
“Ethics must be taken into account when you set up a study,” Plotkin said. “Can I ethically agree to having people acquire the disease because they receive a placebo?”
Precisely.
Moreover, as vaccine scientist and advocate Dr. Paul Offit pointed out, even doing RCTs for new vaccines against diseases for which there is currently no safe and effective vaccine has an ethical price. Pointing to the original RCT of Jonas Salk’s original vaccine against polio in the 1950s, he called it the “casual cruelty of placebo-controlled trials,” because 34 of the 36 children paralyzed by polio in that study were in the placebo group.
Over on Threads, Dr. David Higgins provided some comparisons to give you an idea why requiring placebo-controlled RCTs of second generation vaccines instead of the case now, trials comparing the new vaccine to existing vaccine, is so off-base:


View on Threads
I’ll also point out that the way that HHS emphasizes “inert” placebos is another antivax trope. To the likes of RFK Jr., Del Bigtree, and Aaron Siri, the only acceptable placebo is saline. This emphasis of “inert”—cough, cough, saline!—placebos is another strategy to falsely delegitimize the RCTs used to license some vaccines because, for example, some RCTs use something other than saline. For example, the original HPV vaccine trial used a placebo that contained the same aluminum adjuvant as the vaccine, because it made it very difficult for subjects and their doctors to see any difference in local skin reactions between placebo and vaccine. Sometimes vaccine trials will use the diluent used for the vaccine, minus the antigen(s) used to provoke the immune response, in order to make the two as similar as possible other than the “stuff” that provokes immunity.
The real purposes of the HHS vaccine placebo diktat
With that background in mind, I turn my attention to what I view as the real purpose of the HHS diktat that, henceforth, all new vaccines will be tested against “inert” placebo before they can be licensed. You won’t be surprised to learn that my conclusion is that the purpose of this diktat is, first of all, to suggest that the current CDC childhood vaccination schedule is somehow unsafe and not evidence- or science-based, when it most certainly is. Again, the “tell” is how the HHS statement claims that none of the current vaccines, save COVID-19 vaccines, have been tested in what RFK Jr. considers “real” placebo-controlled RCTs. If HHS had simply said that all new vaccines will be tested against inert placebo, I probably would have shrugged my shoulders and said, in essence, “That’s how they’ve always been tested.” HHS stating that vaccines in the current childhood vaccine schedule tells me that this diktat is at least about casting doubt on the safety and efficacy of childhood vaccines.
I’d argue, though, that the HHS diktat has a more sinister, more explicitly antivax purpose that will depend upon how HHS operationalizes it and defines “new vaccines.” Multiple actions and statements by HHS made me suspect this more sinister purpose. The first occurred month ago, when the date of April 1 came and went without the reformulated version of Novavax being fully licensed, as previously planned by FDA before President Donald Trump and HHS Secretary RFK Jr. took over. Worse, in a highly unusual step, the FDA is now requiring that the new Novavax formulation go through a large RCT before licensure. The new FDA Commissioner, Dr. Marty Makary, who has sworn up and down during the pandemic that he isn’t antivax, is fully on board as well, even to the point of parroting RFK Jr.’s line of “gold standard science”:
Question: if a new formulation is required every year (as is the case with flu vaccines), will new double blind randomized trials be required as if it is a totally new product with no knowledge about it?

Even if by the time the trial is complete, the information is obsolete…
— Prof Jeffrey S Morris (@jsm2334) April 26, 2025
Note how Makary capitalized “Gold Standard of Science.” Ugh.
Or, as Tara Smith put it:
Not to mention that requiring such an RCT will be insanely costly and therefore make it financially not worth it to try to. apply to the FDA to license this vaccine in the US. That’s the idea.
This happened shortly after it looked as though the FDA was going to grant full licensure to the new Novavax formulation, after all. Worse:
“It’s now been years since COVID has presented the threat it once did, and the urgency to rush approval of boosters without normal oversight no longer exists,” said Andrew Nixon, a Health and Human Services spokesman, in a statement.
Prof. Morris is correct. The only difference between the updated Novavax vaccine and the original version of the vaccine is that the company is updating the COVID-19 strains in the vaccine to reflect better what is circulating now. In addition, given that the mRNA vaccines have been treated the same way as Pfizer and Moderna have updated the strains included in them to reflect better the COVID-19 strains in circulation, does the FDA plan on applying a similar logic to COVID-19 vaccines? Note that the administration is using the claim that COVID-19 is supposedly no longer a threat as justification for this move.
Interestingly, in response to concerns about whether or not this precedent will be applied to influenza vaccines, HHS responded:
Nixon, the HHS spokesman, suggested the policy might not apply to the flu shot, “which has been tried and tested for more than 80 years.”
This sounds to me like nothing more than a sop to the old guard, or something that Dr. Makary can say for now in order to reassure himself that he isn’t just a figurehead carrying out RFK Jr.’s assault on vaccines in the US. (Hint: He most definitely is nothing more than a useful idiot chosen to serve as the face of the dismantling the FDA, no matter how much he tries to deny it.) Under RFK Jr. and the spineless Makary, I have little doubt that reasons will be found to expand this requirement for RCTs for “new vaccines” to seasonal flu vaccines by simply redefining each season’s flu vaccine as a “new vaccine.” That’s all it would take to make flu vaccines unavailable in the US, and I have little doubt that this is part of RFK Jr.’s endgame. Indeed, my prediction is that the delicensing of vaccines begins with Novavax, will expand to include mRNA vaccines against COVID-19, and then further expand to end seasonal flu vaccines. Think I’m exaggerating? The administration has arguably been at war with mRNA technology since Trump was inaugurated, as evidenced by the NIH defunding grants to study the technology and the promised increased scrutiny on such vaccines. As for flu vaccines, these have long been the target of antivaxxers as the “low-hanging fruit,” given how their efficacy varies so much from year to year based on how accurately scientists are each year at predicting in the winter the strains that will be circulating the following fall and winter.
The second thing that made me suspect a more sinister purpose to this diktat has been RFK Jr.’s characterization of current vaccine safety monitoring systems. For example, a Fox News story about the “placebo only” diktat, included this statement about vaccine safety monitoring:
HHS also claimed that the CDC has not been monitoring vaccine complications adequately, and criticized its current surveillance system.
“The CDC’s former practice of suppressing information about vaccine injuries has badly eroded trust in our public health agencies,” the agency said.
“The CDC’s own research has shown that the post-licensure surveillance system, VAERS, captures fewer than 1% of vaccine injuries,” the HHS spokesperson said. “It’s a system that was designed to fail. The Vaccine Safety Datalink (VSD) — intended as a backup to VAERS — is virtually unusable for serious research. Both systems have become templates of regulatory malpractice.”
Antivaxxers have frequently attacked the Vaccine Adverse Events Reporting System (VAERS) database and/or weaponized it to give a false picture of the safety of vaccines. Indeed, the weaponization of VAERS against COVID-19 vaccines started almost before COVID-19 vaccines had received emergency use authorization (EUA) in December 2020. (Read here for details. It’s a subject that I’ve written about more times than I can remember, going back to 2006.)
But what about VSD? Why is RFK Jr. attacking the Vaccine Safety Datalink (VSD), which, if you believe what he says about a vaccine safety monitoring system, should be about as close to the “Gold Standard Science” that he claims to want? After all, unlike VAERS, VSD is an active, rather than passive reporting system, that surveys the electronic health records of several large health care organizations, including Kaiser Permanente in California. Last week, a story was published in NBC News, The little-known database at the heart of Kennedy’s vaccine conspiracy theory, that gives an indication why:
Kennedy now oversees the Centers for Disease Control and Prevention — including the Vaccine Safety Datalink — and one of his first initiatives as health secretary was to launch his long-dreamed-of study using the VSD to investigate the link between childhood vaccines and autism. It’s a theory that has already been disproven in dozens of studies, many using VSD data.
“We’re going to be able to get into these databases and give answers to the American public,” Kennedy said last month. He told President Donald Trump at recent Cabinet meeting that he would reveal the cause of autism by September.
But:
Without evidence, anti-vaccine lawyer Aaron Siri and activist Del Bigtree have claimed on the internet show “The HighWire” that the CDC scattered the vaccine safety data after Kennedy took office, making it unavailable for Kennedy’s team to examine. The goal, according to Siri, was to “thwart the ability for the current administration to actually conduct a study in the VSD.”
Not exactly:
There’s one thing that Bigtree and Siri have right: The data in the VSD is not actually housed at the CDC — but that’s not a recent change.
The VSD is a collaboration between a small team at the CDC’s Immunization Safety Office and 11 private health care organizations. Since the 1990s, it’s been used to monitor vaccine safety and conduct studies of rare side effects.
“These studies that use the VSD are able to tell the difference between a condition that coincidentally happens after vaccination and a condition that may actually be the result of vaccination,” said Dr. Joshua Sharfstein, vice dean for public health practice at the Johns Hopkins Bloomberg School of Public Health.
The health care organizations used to send their anonymized medical data to the CDC each year for analysis, but since 2001, the organizations have kept the data on their own servers, to ensure it stays secure.
That’s what’s likely upsetting RFK Jr. He can’t snap his fingers and have immediate access to the data in VSD to torture until it confesses that vaccines cause autism because the data are stored not on CDC or FDA servers, but on the servers of the individual healthcare organizations that contribute to the VSD. Worse for him, although outside researchers can get access to VSD data, they have to go through the institutional review board of Kaiser Permanente, which, unlike the CDC, NIH, and FDA is not controlled by RFK Jr. and therefore can’t be tampered with. You might remember that in 2003 Mark and David Geier, the latter of whom was hired by NIH to do RFK Jr.’s study to look at vaccines as a risk factor for autism, got busted for trying to merge datasets in a manner that could compromise protected health information (PHI) in violation of HIPAA and to copy some of the data, both of which violated their IRB-approved protocol. The news story notes that David Geier has viewed that as “oppression” and “coverup”:
For the Geiers, the experience was one of oppression. At anti-vaccine conferences, they presented PowerPoints alleging that the CDC had “restricted” their access to data and “attacked” their findings.
At the same conferences, Kennedy, who has consistently praised the Geiers, told a similar story.
“Oh, they’ve hidden it, and they won’t let anybody in it, except their own guys who cherry-pick and design these fabricated studies and change the protocols constantly to try to use it to defend vaccines,” Kennedy said in a 2017 keynote at AutismOne, an anti-vaccine conference for parents of autistic children where the Geiers also presented. “It’s being used instead to craft these fabricated, fraudulent studies,” he said, “to fool the public about vaccine safety.”
Far be it from me to say that better postlicensure safety monitoring for vaccines—for all pharmaceuticals, really—wouldn’t be desirable and an objectively good thing. (Indeed, I’ve argued the opposite.) However, also far be it from me to fall for RFK Jr.’s claim that all he wants to do is to improve vaccine safety monitoring. Indeed, he is the last person I would put in charge of vaccine safety monitoring; sadly, the same was not true of President Trump, and here we are, with RFK Jr. overseeing the agencies that are in fact responsible for vaccine safety monitoring. Until RFK Jr. is no longer in charge, the best that can be done is to try to protect, as much as possible, the integrity of existing vaccine safety monitoring systems. In any event, it’s clear that what RFK Jr. really wants is access to the data to cherry pick in a manner to “prove” a connection between autism and vaccines or to blame vaccines for all the things that antivaxxers like to attribute falsely to vaccines.
Finally, the last (or at least he most recent) tell that RFK Jr. is working to eliminate vaccines also appeared news stories with titles like Kennedy Orders Search for New Measles Treatments Instead of Urging Vaccination.
Basically:
“Secretary Kennedy will be enlisting the entire agency to activate a scientific process to treat a host of diseases, including measles, with single or multiple existing drugs in combination with vitamins and other modalities,” the HHS spokesperson said in a statement to CBS News.
The statement said the CDC effort would involve working with universities “to develop protocols, conduct testing, and pursue approval for new uses of safe and effective therapeutics that meet the highest scientific standards.”
While the CDC is continuing to recommend vaccination “as the most effective way to prevent the disease,” the statement says they recognize some Americans “may choose not to vaccinate.”
“Our commitment is to support all families — regardless of their vaccination status — in reducing the risk of hospitalization, serious complications, and death from measles,” the statement said.
Unfortunately, as is the case with most viral diseases, the only treatment for patients who have been infected is supportive care. True, in children who are malnourished—which is not the majority of American children—treatment with vitamin A could have some beneficial effects but is useless for most children, but nothing beats vaccination to prevent the disease in the first place. While it might sound benevolent and high-minded to say that he’s supporting “all families regardless of vaccination status,” what RFK Jr. is really doing is, as he’s been doing all along, paying lip service to the MMR vaccine (grudgingly) while deemphasizing it in favor of quackery and unproven treatments like the steroid inhaler budesonide. It’s the same playbook that COVID-19 deniers used with COVID-19: Tout hydroxychloroquine and ivermectin, neither of which have any therapeutic activity against COVID-19, the implication being that you don’t need to get vaccinated because there is an effective, safe, and inexpensive cure that you can use if you contract the disease.
The future looks grim
As I was writing this, it was difficult not to note that, at 935 cases, the number of measles cases in the current outbreak continues to climb. Fortunately, there have been no further deaths (yet) since the second child died a month ago. However, the number of children who have died of influenza thus far has hit 216, the most since the H1N1 pandemic in 2009-2010, thanks to declining vaccination rates. That’s not all, however. The number of whooping cough cases is also skyrocketing, with 8,400 cases already reported thus far, which is double what was reported this time last year, which was double what was reported in 2023 at this time. (In my own state of Michigan, cases have surged by nearly 1,800%.
As I like to say, measles tends to be the canary in the coal mine when it comes to declining vaccine uptake, if only because it’s the most contagious. It almost always comes back first, because the highest level of population immunity (a.k.a. herd immunity) is required to keep it in check. Then, as vaccination rates continue to fall, next up we would expect to see the return of pertussis, which is also very contagious and for which the vaccine is not as effective. Then we have influenza, for which not enough children were vaccinated even before the pandemic resulted in the turbocharging of antivax misinformation and conspiracy theories. it was also just last week that I discussed a modeling study that shows the catastrophic possibilities if vaccination rates continue to fall, with measles becoming endemic and even polio returning. Even if RFK Jr. were removed and replaced with a pro-vaccine HHS Secretary today, sadly, I fear that a huge resurgence of vaccine-preventable diseases is inevitable, given how much time it will take to reverse the mistrust fueled by the pandemic and the devastation that RFK Jr. has already in less than three months inflicted on the CDC and FDA.
But wait, I hear RFK Jr. apologists saying, what about the new $500 million next-generation universal vaccine platform announced last week? Let’s just say that it’s a lot less than it seems:
That HHS and NIH are going to continue to invest in efforts to make vaccines that could reduce the impact of future pandemics was good news, many thought. But why spend so much money on using whole killed viruses — an approach pioneered in the last century — as the basis of the vaccines?
“I was confused by the messaging, because it is a 70-year-old technology,” said one scientist who works in vaccine development, noting this was the method used by Jonas Salk to create the world’s first polio vaccine in the early 1950s.
Another scientist familiar with the scope of research that is being done to try to develop a so-called universal flu vaccine to protect against a range of dangerous strains — like H5N1 bird flu — was blunt. “There is incredible work going on. This is not it.”
Of course, our new NIH Director Dr. Jay Bhattacharya portrays using whole-virus vaccines as a “preserve the virus’s structural integrity while eliminating infectivity,” even though whole virus vaccines were how flu vaccines were made for decades before newer technologies arose, and the influenza virus still managed to evolve quite handily to avoid preexisting immunity. Truly, everything old is new again, as is the likely grift, given that this project is totally in-house and therefore bypasses traditional peer review, likely to funnel money to favored researchers. Color me unimpressed; in reality this initiative strikes me as a way of deflecting from the rejection of the mRNA platform that produced COVID-19 viruses.
A couple of months ago, I wrote a post speculating on what RFK Jr. might do to undermine the US vaccination system at the federal level, and, make no mistake, that is what he is about, his lies that he is “not antivaccine” and only wants to improve trust and make vaccines safer notwithstanding. Disturbingly, he’s actually doing more than I had predicted. Although I foresaw, for instance, his changing CDC—and HHS—messaging on vaccines, meddling with the Advisory Committee on Immunization Practices, promoting research aligned with antivaxxers, and the like, I should have foreseen this shift to requiring placebo-controlled trials for basically everything.
I fear that this move is a prelude that lays the groundwork for RFK Jr. to start delicensing vaccines. He’ll start first by refusing to approve any new COVID-19 vaccines without large placebo-controlled RCTs, and then likely move on to doing the same for flu vaccines, probably not this year but next year or the year after. In the meantime, he’ll look at vaccines on the childhood schedule for ways he can justify delicensing them—or at least taking them off the CDC’s recommended vaccine schedule so that insurance is no longer required to pay for them—and requiring new, large RCTs to license them again, likely starting with HPV vaccines and the birth dose of hepatitis B vaccine. He’ll weaponize VAERS and, if he can get his hands on it, VSD, in order to create apparent adverse reactions attributable to specific vaccines, in particular autism, in order to justify delicensing them and requiring new, long term RCTs to relicense them. In the meantime, vaccine-preventable diseases will come roaring back, as they always do whenever vaccination rates fall. Children will die, as well adults. I hope I’m wrong, but I really do fear that this is what RFK Jr.’s “make America healthy again” will mean in practice.



   Of SBM and EBM Redux. Part I: Does EBM Undervalue Basic Science and Overvalue RCTs?
Kimball Atwood
Link
During the most recent kerfuffle about whether or not Evidence-Based Medicine can legitimately claim to be science-based medicine, it became clear to me that a whole, new round of discussion and documentation is necessary. This is frustrating because I’ve already done it several times, most recently less than a year ago. Moreover, I’ve provided a table of links to the whole series at the bottom of each post*…Never mind, here goes, and I hope this will be the last time it is necessary because I’ll try to make this the “go to” series of posts for any future such confusions. 
The points made in this series, most of which link to posts in which I originally made them, are in response to arguments from statistician Steve Simon, whose essay, Is there something better than Evidence Based Medicine out there?, was the topic of Dr. Gorski’s rebuttal on Monday of this week, and also from several of the comments following that rebuttal. Mr. Simon has since revised his original essay to an extent, which I’ll take into account. I’ll frame this as a series of assertions by those who doubt that EBM is deficient in the ways that we at SBM have argued, followed in each case by my response.
First, a disclaimer: I don’t mean to gang up on Mr. Simon personally; others hold opinions similar to his, but his essay just happens to be a convenient starting point for this discussion. FWIW, prior to this week I perused a bit of his blog, after having read one of his comments here, and found it to be well written and informative. 
What’s in a Name?
One of Mr. Simon’s objections, in his revision, is this:
What is SBM? Here’s a definition found on the opening entry in the SBM blog:
“the use of the best scientific evidence available, in the light of our cumulative scientific knowledge from all relevant disciplines, in evaluating health claims, practices, and products.” http://www.sciencebasedmedicine.org/?p=1
But how does this differ from David Sackett’s definition of EBM?
“the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” http://www.bmj.com/content/312/7023/71.full
The only substantial difference I see is the adjective “scientific” that appears twice in the definition of SBM. The claim on the SBM blog is that EBM ignores scientific plausibility. Actually, ignores is too strong a word.
“EBM ‘levels of evidence’ hierarchy renders each entry sufficient to trump those below it. Thus a ‘positive’ clinical trial is given more weight than ‘physiology, bench research or “first principles”,’ even when the latter definitively refute the claim.” http://www.sciencebasedmedicine.org/?p=42
(I agree that “ignore” is too strong a word, but I didn’t actually write it that way, as Dr. Gorski pointed out and as I think Mr. Simon was acknowledging above.)
A difference between Sackett’s definition and ours is that by “current best evidence” Sackett means the results of RCTs. I realize that this assertion requires documentation, which will come below. A related issue is the definition of “science.” In common use the word has at least three, distinct meanings: 1. The scientific pursuit, including the collective institutions and individuals who “do” science; 2. The scientific method; 3. The body of knowledge that has emerged from that pursuit and method (I’ve called this “established knowledge”; Dr. Gorski has called it “settled science”).
I will argue that when EBM practitioners use the word “science,” they are overwhelmingly referring to a small subset of the second definition: RCTs conceived and interpreted by frequentist statistics. We at SBM use “science” to mean both definitions 2 and 3, as the phrase “cumulative scientific knowledge from all relevant disciplines” should make clear. That is the important distinction between SBM and EBM. “Settled science” refutes many highly implausible medical claims—that’s why they can be judged highly implausible. EBM, as we’ve shown and will show again here, mostly fails to acknowledge this fact.
Finally, Mr. Simon has misinterpreted our goal at SBM:
But if someone wants to point out that EBM needs work, I’m fine with that. I dislike that they think that EBM needs to be replaced with something better.
You see EBM as being wrong often enough that you see value in creating a new label, SBM. I see SBM as being that portion of EBM that is being done thoughtfully and carefully, and don’t see the need for a new label.
I generally bristle when people want to create a new and improved version of EBM and then give it a new label.
I am as harshly critical of the hierarchy of evidence as anyone. I see this as something that will self-correct over time, and I see people within EBM working both formally and informally to replace the rigid hierarchy with something that places each research study in context. I’m staying with EBM because I believe that people who practice EBM thoughtfully do consider mechanisms carefully. That includes the Cochrane Collaboration.
Mr. Simon, we agree! Yes, we are pointing out that EBM needs work. Yes, SBM is that (tiny) portion of EBM that is being done thoughtfully and carefully, and if it were mainly done that way there would be no need to call attention to the point. Our goal is not to change the name of EBM (“give it a new label”). Our goal is to convince EBM to live up to its current name. Yes, it may self-correct over time, but we are trying to shorten that time. Bad things have unnecessarily happened, in part due to EBM’s scientific blind spot: As currently practiced, it doesn’t rationally consider all the evidence. We don’t see much evidence that people at the highest levels of EBM, eg, Sackett’s Center for EBM or Cochrane, are “working both formally and informally to replace the rigid hierarchy with something that places each research study in context.”
We chose to call our blog “science-based medicine” only because the term “evidence-based medicine” had already been taken, and we needed to distinguish ourselves from the inaccurate use of the word “evidence” in “EBM.” I’ve written about this before, and have made the point utterly clear:
These are the reasons that we call our blog “Science-Based Medicine.” It is not that we are opposed to EBM, nor is it that we believe EBM and SBM to be mutually exclusive. On the contrary: EBM is currently a subset of SBM, because EBM by itself is incomplete. We eagerly await the time that EBM considers all the evidence and will have finally earned its name. When that happens, the two terms will be interchangeable.
Plausibility Misinterpreted
Mr. Simon’s interpretation of our view of plausibility, like that of many others, is wrong:
I would argue further that it is a form of methodolatry to insist on a plausible scientific mechanism as a pre-requisite for ANY research for a medical intervention. It should be a strong consideration, but we need to remember that many medical discoveries preceded the identification of a plausible scientific mechanism.
I think, from his revision, that Mr. Simon understood Dr. Gorski’s explanation of why this was wrong, but I’m not certain. The misrepresentation of scientific plausibility is an issue that I’ve faced for years, as explained previously here:
Plausibility ≠ Knowing the Mechanism
Let’s first dispense with a simple misunderstanding: We, by which I mean We Supreme Arbiters of Plausibility (We SAPs) here at SBM, do not require knowing the mechanism of some putative effect in order to deem it plausible. This seems so obvious that it ought not be necessary to repeat it over and over again, and yet the topic can’t be broached without some nebbishy South Park do-gooder chanting a litany of “just because you don’t know how it works doesn’t mean it can’t work,” as if that were a compelling or even relevant rebuttal. Let’s get this straight once and for all: IT ISN’T.
Steve Novella explained why at the Yale conference and again here. We talked about it at TAM7 last summer. For a particularly annoying example, read the three paragraphs beginning with “Mr. Gagnier’s understanding of biological plausibility” here.
OK, I’ll admit that I’m beginning to learn something from such frustration. Perhaps we’ve not been so good at explaining what we mean by plausibility. The point is not that we don’t know a particular mechanism for homeopathy, for example; the point is that any proposed mechanism would necessarily violate scientific principles that rest on far more solid ground than any number of equivocal, bias-and-error-prone clinical trials could hope to overturn. The same is true for “energy medicine” and for claims based on non-existent anatomical structures (iridology, reflexology, auricular acupuncture, meridians, chiropractic “subluxations”), non-existent physiologic functions (“craniosacral rhythms“), or non-existent anatomic-physiologic relations (“neurocranial restructuring,” “detoxification” with coffee enemas, dissolving tumors with orally administered pancreatic enzymes). The spectrum of implausible health claims euphemistically dubbed “CAM” is full of such nonsense.
Reader daedalus2u proposed a useful way to clarify the point:
I think the idea of prior plausibility should actually be reframed into one of a lack of prior implausibility. It isn’t that one should have reasons to positively think that something is plausible before testing it, but rather that one should not be able to come up with reasons (actually data) why it is fatally implausible.
Some of what We deem implausible will not be fatally so, of course. Implausibility can be based not only on established physical and biological knowledge, but also on studies, as is the case for sticking needles into people, injecting them with chelating agents, or claiming that autism is caused by childhood immunizations.
EBM, Basic Science, and RCTs
Steve Simon wrote, “I have not seen any serious evidence of EBM relying exclusively on RCTs. That’s certainly not what David Sackett was proposing in the 1996 BMJ editorial…” And: “No thoughtful practitioner of EBM, to my knowledge, has suggested that EBM ignore scientific mechanisms.”
Want serious evidence? Consider these quotations from Cochrane reviews, originally posted here:
In view of the absence of evidence it is not possible to comment on the use of homeopathy in treating dementia.
There is not enough evidence to reliably assess the possible role of homeopathy in asthma. As well as randomised trials, there is a need for observational data to document the different methods of homeopathic prescribing and how patients respond.
There is currently little evidence for the efficacy of homeopathy for the treatment of ADHD. Development of optimal treatment protocols is recommended prior to further randomised controlled trials being undertaken.
Though promising, the data were not strong enough to make a general recommendation to use Oscillococcinum for first-line treatment of influenza and influenza-like syndromes. Further research is warranted but the required sample sizes are large.
Yes, EBM undervalues basic science and overvalues RCTs when the former is sufficient to reject a claim. EBM also undervalues experimental evidence other than RCTs when the former is sufficient to reject a claim, as will be discussed. Here is how a truly evidence-based review might conclude a discussion of homeopathy for dementia:
The probability that homeopathy is specifically therapeutic for dementia is, for all practical purposes, zero.
The following is from my first post on the topic, in which I reviewed the overwhelming evidence—from basic science and pre-clinical research—that homeopathic ‘remedies’ have no, specific therapeutic actions, and wondered why the most esteemed exponents of EBM have written that such treatments are “promising” and that “further randomized trials are needed.” I included the Center for Evidence-based Medicine’s formal “Levels of Evidence” scheme (not copied here), the pertinent quotation from Sackett’s 1996 editorial, my opinion that this failure of EBM was initially unintended, how Sackett et al eventually did address “CAM,” and the Cochrane abstracts quoted above:
It wasn’t meant to be like this. When I first discussed with my fellow bloggers the curious absence of established knowledge in the EBM “levels of evidence” hierarchy, at least one insisted that this could not be true, and in a sense he was correct. David Sackett and other innovators of EBM do include basic science in their discussions, but they recommend invoking it only when there are no clinical trials to consider:
Evidence based medicine is not restricted to randomised trials and meta-analyses. It involves tracking down the best external evidence with which to answer our clinical questions…And sometimes the evidence we need will come from the basic sciences such as genetics or immunology. It is when asking questions about therapy that we should try to avoid the non-experimental approaches, since these routinely lead to false positive conclusions about efficacy. Because the randomised trial, and especially the systematic review of several randomised trials, is so much more likely to inform us and so much less likely to mislead us, it has become the “gold standard” for judging whether a treatment does more good than harm.
That statement is consistent with EBM’s formal relegation of established knowledge to “level 5,” as seen in the Figure. I am not a historian of EBM and don’t care to be, but I suspect that the explanation for this choice is that “they never saw ‘CAM’ coming.” In other words, it probably didn’t occur to Sackett and other EBM pioneers that anyone would consider performing clinical trials of methods that couldn’t pass the muster of scientific plausibility. Their primary concern was to emphasize the insufficiency of basic science evidence in determining the safety and effectiveness of new treatments. In that they were quite correct, but trials of “CAM” have since reminded us that although established knowledge may be an insufficient basis for accepting a treatment claim, it is still a necessary one.
Take note: Sackett wrote, “we should try to avoid the non-experimental approaches, since these routinely lead to false positive conclusions about efficacy.” My point is that pre-RCT evidence does not routinely (if ever) lead to false negative conclusions. In that passage, moreover, Sackett seems to suggest that the only alternative to a “non-experimental approach” is an RCT; yet there are often other types of experiments that can definitively refute treatment claims, as will be discussed. Eventually Sackett et al did catch wind of “CAM,” but they got it exactly wrong:
Lacking that perspective, Sackett’s Center for Evidence-Based Medicine promulgates an “Introduction to evidence-based complementary medicine” by “CAM” researcher Andrew Vickers. There is not a mention of established knowledge in it, although there are references to several claims, including homeopathy, that are refuted by things that we already know. Vickers is also on the advisory board of the Cochrane CAM Field, along with Wayne Jonas and several other “CAM” enthusiasts.
In another post I cited the 2006 Cochrane Review of Laetrile:
A 2006 Cochrane Review of Laetrile for cancer would, if its recommendations were realized, stand the rationale for RCTs on its head:
The most informative way to understand whether Laetrile is of any use in the treatment of cancer, is to review clinical trials and scientific publications. Unfortunately no studies were found that met the inclusion criteria for this review.
Authors’ conclusions
The claim that Laetrile has beneficial effects for cancer patients is not supported by data from controlled clinical trials. This systematic review has clearly identified the need for randomised or controlled clinical trials assessing the effectiveness of Laetrile or amygdalin for cancer treatment.
Why does this stand the rationale for RCTs on its head? A definitive case series led by the Mayo Clinic in the early 1980s had overwhelmingly demonstrated, to the satisfaction of all reasonable physicians and biomedical scientists, that not only were the therapeutic claims for Laetrile baseless, but that the substance is dangerous. The subjects did so poorly that there would have been no room for a meaningful advantage in outcome with active treatment compared to placebo or standard treatment… The Mayo case series “closed the book on Laetrile,” the most expensive health fraud in American history at the time, only to have it reopened more than 20 years later by well-meaning Cochrane reviewers who seemed oblivious of the point of an RCT.
Is that review not serious evidence that the Cochrane Collaboration overvalues RCTs? In this case, moreover, it wasn’t only basic science that Cochrane ignored, but a definitive piece of clinical research that was not an RCT. Sure, I know that Cochrane is not the only pinnacle of EBM, but it’s one of them.
In both that post and another, I called attention to a statement that Edzard Ernst, the most prolific EBM-style “CAM” researcher of the past 20 years, had made in 2003:
A couple of years ago I was surprised to find that one of the authors of [the Cochrane Laetrile] review was Edzard Ernst, a high-powered academic who over the years has undergone a welcomed transition from cautious supporter to vocal critic of much “CAM” research and many “CAM” methods. He is now a valuable member of our new organization, the Institute for Science in Medicine, and we are very happy to have him. I believe that his belated conversion to healthy skepticism was due, in large part, to his allegiance to the formal tenets of EBM. I recommend a short debate published in 2003 in Dr. Ernst’s Focus on Alternative and Complementary Therapies (FACT), pitting Jacqueline’s countryman Cees Renckens against Dr. Ernst himself. Dr. Ernst responded to Dr. Renckens’s plea to apply science to “CAM” claims with this statement:
In the context of EBM, a priori plausibility has become less and less important. The aim of EBM is to establish whether a treatment works, not how it works or how plausible it is that it may work. The main tool for finding out is the RCT. It is obvious that the principles of EBM and those of a priori plausibility can, at times, clash, and they often clash spectacularly in the realm of CAM.
I’ve discussed that debate before on SBM, and I consider it exemplary of what is wrong with how EBM weighs the import of prior probability. Dr. Ernst, if you are reading this, I’d be interested to know whether your views have changed. I hope that you no longer believe that human subjects ought to be submitted to a randomized, controlled trial of Laetrile!
Uh, talk about “suggesting that EBM ignore scientific mechanisms”! When the principles of EBM and those of a priori plausibility clash spectacularly in the realm of CAM, it is a priori plausibility that should take precedence—not merely because the latter renders RCTs unnecessary, but because for such questions RCTs tend to confuse rather than clarify, as will be discussed further in the next part of this series.
I am happy to report that Dr. Ernst wrote me privately about that passage, with the answer that I’d mostly hoped for:
Have I changed my mind? I am not as sure as the sceptics seem to be that I ever was a supporter of CAM and I am still a bit sceptic about the sceptics [which perhaps makes me the "ueber-sceptic"]. Would I argue for more Laetrile studies? NO.
Even more to the point, perhaps, is a recent editorial by Dr. Ernst in which he calls homeopathy “absurd” and compares it to other, obvious absurdities, which I doubt he’d have done only a few years ago:
Should we keep an open mind about astrology, perpetual motion, alchemy, alien abduction, and sightings of Elvis Presley? No, and we are happy to confess that our minds have closed down on homeopathy in the same way.
This kind of clear thinking, as easy as it ought to be for intelligent people, seems oddly difficult for those steeped in EBM. I’ll offer another example in part 2, as part of my response to Mr. Simon’s assertion that “There is some societal value in testing therapies that are in wide use, even though there is no scientifically valid reason to believe that those therapies work.”

   Of SBM and EBM Redux. Part II: Is it a Good Idea to test Highly Implausible Health Claims?
Kimball Atwood
Link
Review
This is the second post in a series* prompted by an essay by statistician Stephen Simon, who argued that Evidence-Based Medicine (EBM) is not lacking in the ways that we at Science-Based Medicine have argued. David Gorski responded here, and Prof. Simon responded to Dr. Gorski here. Between that response and the comments following Dr. Gorski’s post it became clear to me that a new round of discussion would be worth the effort.
Part I of this series provided ample evidence for EBM’s “scientific blind spot”: the EBM Levels of Evidence scheme and EBM’s most conspicuous exponents consistently fail to consider all of the evidence relevant to efficacy claims, choosing instead to rely almost exclusively on randomized, controlled trials (RCTs). The several quoted Cochrane abstracts, regarding homeopathy and Laetrile, suggest that in the EBM lexicon, “evidence” and “RCTs” are almost synonymous. Yet basic science or preliminary clinical studies provide evidence sufficient to refute some health claims (e.g., homeopathy and Laetrile), particularly those emanating from the social movement known by the euphemism “CAM.”
It’s remarkable to consider just how unremarkable that last sentence ought to be. EBM’s founders understood the proper role of the rigorous clinical trial: to be the final arbiter of any claim that had already demonstrated promise by all other criteria—basic science, animal studies, legitimate case series, small controlled trials, “expert opinion,” whatever (but not inexpert opinion). EBM’s founders knew that such pieces of evidence, promising though they may be, are insufficient because they “routinely lead to false positive conclusions about efficacy.” They must have assumed, even if they felt no need to articulate it, that claims lacking such promise were not part of the discussion. Nevertheless, the obvious point was somehow lost in the subsequent formalization of EBM methods, and seems to have been entirely forgotten just when it ought to have resurfaced: during the conception of the Center for Evidence-Based Medicine’s Introduction to Evidence-Based Complementary Medicine.
Thus, in 2000, the American Heart Journal (AHJ) could publish an unchallenged editorial arguing that Na2EDTA chelation “therapy” could not be ruled out as efficacious for atherosclerotic cardiovascular disease because it hadn’t yet been subjected to any large RCTs—never mind that there had been several small ones, and abundant additional evidence from basic science, case studies, and legal documents, all demonstrating that the treatment is both useless and dangerous. The well-powered RCT had somehow been transformed, for practical purposes, from the final arbiter of efficacy to the only arbiter. If preliminary evidence was no longer to have practical consequences, why bother with it at all? This was surely an example of what Prof. Simon calls “Poorly Implemented Evidence Based Medicine,” but one that was also implemented by the very EBM experts who ought to have recognized the fallacy.
There will be more evidence for these assertions as we proceed, but the main thrust of Part II is to begin to respond to this statement from Prof. Simon: “There is some societal value in testing therapies that are in wide use, even though there is no scientifically valid reason to believe that those therapies work.”
Some such Testing is Useful (and Fun)…
First, let me say that I am not opposed to all trials pertaining to such methods (not “therapies,” which begs the question), assuming that the risks to subjects are minimal, the funding is not public, and the study is honest and ethical in every respect. For example, I’m happy that studies have been done looking at interexaminer reliability of practitioners who claim to detect ‘craniosacral rhythms’ (there is none), at the ability of ‘therapeutic touch’ practitioners to detect the ‘human energy field’ when denied visual cues (they can’t), or whether ‘provers’ can distinguish between an ‘ultramolecular’ homeopathic preparation and a ‘placebo’ (they can’t).
Those sorts of trials are small, cheap, paid for with private money, often test the claimants themselves (on whom the onus of proof belongs), have minimal risk of harm or discomfort, and each hypothesis tested is a sine qua non of a larger therapeutic claim. Such tests are simpler and less bias- and error-prone than are efficacy trials of the corresponding claims, and are sufficient to reject those claims. Yet EBM typically ignores such research when reviewing efficacy claims, as exemplified by the Cochrane homeopathy abstracts quoted in Part I and by a Cochrane review of “touch therapies.” In the case of homeopathy, there are several other testable hypotheses that, when tested, have also disconfirmed the larger claim. Why aren’t they cited in EBM-style reviews?
Here I must give Prof. Simon some credit. In his most recent discussion of EBM vs. SBM he wrote the following:
Now part of me says things like, no funding of research into therapeutic touch until someone can replicate the Emily Rosa experiment and show different results than Ms. Rosa did. So I’m kind of split on this issue.
It was the Emily Rosa experiment that demonstrated that ‘therapeutic touch’ practitioners could not detect the ‘human energy field’ when denied visual cues. Thus in some ways Prof. Simon and I are not that far apart, although I’m not at all split on the issue. I’ll discuss more of this in Part III.
…But Efficacy Trials are Not
Regarding publicly funded efficacy trials of implausible claims, my responses are several, including those that Dr. Gorski has already discussed: such studies don’t convince true believers, they are frequently unethical and even dangerous, and they waste research funds. Prof. Simon counters that to have societal value, studies needn’t convince true believers, only fence-sitters (true but irrelevant—see below), and that the public money spent is such a small portion of the entire health care bill that it makes little difference—but here he stumbles a bit:
Money spent on health care is a big, big pot of money and the money spent on research is peanuts by comparison. If we spend some research money to help insure that the big pot of money is spent well, we have been good stewards of the limited research moneys.
The issue, of course, is whether or not the research money is well spent. In the case of efficacy trials of methods that lack scientific bases, the money is never well spent. The same people who would be convinced by such trials ought to be convinced by NIH scientists simply explaining to them, in a definitive way, that there is no scientifically valid reason for those methods to work, or that the methods have already been disproved by other investigations, including the types of trials just mentioned. If such statements are not convincing, why not? Remember, fence-sitters are not true believers or anti-intellectual, conspiracy-theory-laden, anti-fluoride, pro-Laetrile, pro-chelation, anti-vax paranoiacs. If they were, they wouldn’t be convinced by trials either, would they?
To explain why otherwise reasonable people might not be convinced by definitive statements based on science, we need look no further than EBM’s own scientific blind spot, as perfectly exemplified by Dr. Ernst’s words in his 2003 debate with Cees Renckens, quoted in Part I:
In the context of EBM, a priori plausibility has become less and less important. The aim of EBM is to establish whether a treatment works, not how it works or how plausible it is that it may work. The main tool for finding out is the RCT…
Ironically, it may be that those at most risk for being unconvinced by science are physicians themselves—thanks to EBM. What follows is the passage that I promised at the end of Part I. It illustrates just how elusive clear thinking can be, even for very intelligent people, after they’ve been steeped in EBM. Originally posted here, it also introduces the next reason that we should, er, look askance at calls for efficacy trials of implausible claims:
Failing to consider Prior Probability leads to Unethical Human Studies
An example…was the regrettable decision of two academic pediatricians, one of whom is a nationally-recognized expert in pediatric infectious disease, to become co-investigators in an uncontrolled trial of homeopathic “remedies” for acute otitis media (AOM) in 24 children, ages 8 mos-77 mos. The treating investigators were homeopaths. The report provided evidence that 16 of the children had persistent or recurrent symptoms, lasting from several days to 4 weeks after diagnosis. Nevertheless, no child was prescribed analgesics or anti-pyretics, and only one child was eventually prescribed an antibiotic by an investigator (another in an emergency room). There is no evidence that the investigators evaluated any of the subjects for complications of AOM, nor did the two academic pediatricians “interact with any research subjects.” Similar examples are not hard to find.
Funny thing about EBM’s tenacious hold on medical academics: a few years ago, when I first noticed the report just described, I ran it by a friend who is the chief of pediatrics at a Boston area hospital and a well-known academic pediatrician in his own right. After I explained the tenets of homeopathy, he agreed that it is highly implausible. At that point I expected him to agree that the trial had been unethical. Instead he queried, “but isn’t its being unproven just the reason that they should be allowed to study it?” There was no convincing him.
Such faith in clinical trials as absolute, objective arbiters of truth about claims that contradict established knowledge raises another point that will have to wait for Part III: RCTs are not objective arbiters in such cases, but rather tend to confuse more than clarify. For now, let’s continue to look at…
Human Studies Ethics
A “Clinically Competent Medical Person”
That homeopaths were accepted as the sole, treating clinician/investigators in the trial just mentioned should make any IRB member’s eyebrows raise. According to the Helsinki Declaration,
Medical research involving human subjects should be conducted only by scientifically qualified persons and under the supervision of a clinically competent medical person. The responsibility for the human subject must always rest with a medically qualified person and never rest on the subject of the research, even though the subject has given consent.
The physician may combine medical research with medical care, only to the extent that the research is justified by its potential prophylactic, diagnostic or therapeutic value. When medical research is combined with medical care, additional standards apply to protect the patients who are research subjects.
Dr. Gorski mentioned two other trials that I’ve written extensively about, the Gonzalez trial for cancer of the pancreas and the ongoing Trial to Assess Chelation Therapy (TACT). Each of those claims had a miniscule prior probability, but proponents justified each by “popularity” and by appeals to EBM such as the AHJ editorial quoted above. Each trial involved clinically incompetent investigators chosen by the NIH: Gonzalez himself in the former and numerous chelationists in the latter, most of whom are members of the organizations described here, and many of whom have been subjected to actions by state medical boards, federal civil settlements, or criminal convictions. Predictably, the Gonzalez trial involved unnecessary torture of human subjects, and the TACT has involved unnecessary deaths.
Below are quotations from a post that subjected the Gonzalez trial to ethical scrutiny; most of the arguments apply to implausible claims in general. For the purposes of this post I’ll provide new topic headings and a few comments.
Informed Consent and Clinical Equipoise
In 2003, using the Gonzalez regimen as an example, I argued that the information offered to prospective subjects of trials of implausible claims is likely to be misleading:
Plausibility also figures in informed consent language and subject selection. How many subjects who are not wedded to “alternative medicine” would be likely to join a study that independent reviewers rate as unlikely to yield any useful results, or in which the risks are stated to outweigh the potential benefits? Are informed consents for such studies honest? In at least one case cited in the following paragraph, the answer is “no.” Nor may subjects who prefer “alternative” methods be preferentially chosen for such research even if they seek this, because “fair subject selection requires that the scientific goals of the study, not vulnerability, privilege, or other factors unrelated to the purposes of the research, be the primary basis for determining the groups and individuals that will be recruited and enrolled” (Emanuel et al. 2000).
The Office for Human Research Protections recently cited Columbia University for failure to describe serious risks on the consent form of its “Gonzalez” protocol for cancer of the pancreas, funded by the NCCAM (OHRP 2002). The study proposes to compare the arduous “Gonzalez” method, which is devoid of biological rationale, to gemcitabine, an agent acknowledged by the investigators to effect “a slight prolongation of life and a significant improvement in . . . quality of life.” Nevertheless, a letter from Columbia to prospective subjects states, “it is not known at the present time which treatment approach is best [sic] overall” (Chabot 1999). The claim of clinical equipoise, or uncertainty in the expert medical community over which treatment is superior—necessary to render a comparison trial ethical—is not supported by the facts (Freedman 1987).
The consent forms for both the TACT and the homeopathy trial mentioned above were also uninformative or worse. For my comments on the former, look here under “Comments on the TACT Consent Form”; for the homeopathy trial’s consent form, look here and reach your own conclusions (hint: there is no mention of the risks of omitting standard treatments for acute otitis media).
Ms. Gurney’s article [about a friend who submitted himself to the Gonzalez trial] provides additional, compelling evidence that the Gonzalez protocol did not meet the standard of clinical equipoise:
…at ASCO, I learned quickly and definitively that the Gonzalez protocol was a fraud; no mainstream doctors believed it was anything else and they were surprised that anyone with education would be on it.
The “mainstream doctors” of the American Society of Clinical Oncology must be judged representatives of the pertinent “expert medical community.”
The TACT also violates the principle of clinical equipoise, even as it claims to do otherwise, as discussed here under “‘Clinical Equipoise’ and the Balance for Risks and Benefits.”
Science and Ethics
There is a consensus, among those who consider human studies ethics, that a study must be scientifically sound in order to be ethical. According to the Council for International Organizations of Medical Sciences. International Ethical Guidelines for Biomedical Research Involving Human Subjects (CIOMS; Geneva, Switzerland:1993. Quoted here):
Scientifically unsound research on human subjects is ipso facto unethical in that it may expose subjects to risks or inconvenience to no purpose.
The Helsinki Declaration agrees:
Medical research involving human subjects must conform to generally accepted scientific principles, be based on a thorough knowledge of the scientific literature, other relevant sources of information, and on adequate laboratory and, where appropriate, animal experimentation.
There is no body of basic science or animal experimentation that supports the claims of Gonzalez.
Emanuel and colleagues, writing in JAMA in 2000, asserted:
Examples of research that would not be socially or scientifically valuable include clinical research with…a trifling hypothesis…
I assert that highly implausible claims ought to be viewed as “trifling hypotheses.”
The Fallacy of Popularity
Virtually all of the research agenda of the NCCAM has been justified by the assertion that implausible claims that are popular require research, merely because people are using them. Referring to the opinions of the late NCCAM Director Stephen Straus, Science Magazine wrote in 2000:
Scientific rigor is sorely needed in this enormously popular but largely unscrutinized field….Most of these substances and treatments have not been tested for either safety or efficacy.
As surprising as it may be to some, however, a method’s popularity may not supersede the interests of individual trial subjects. According to the Helsinki Declaration:
In medical research on human subjects, considerations related to the well-being of the human subject should take precedence over the interests of science and society.
The Belmont Report agrees:
Risks and benefits of research may affect the individual subjects, the families of the individual subjects, and society at large (or special groups of subjects in society). Previous codes and Federal regulations have required that risks to subjects be outweighed by the sum of both the anticipated benefit to the subject, if any, and the anticipated benefit to society in the form of knowledge to be gained from the research. In balancing these different elements, the risks and benefits affecting the immediate research subject will normally carry special weight.
The U.S. Code of Federal Regulations is unequivocal:
The IRB should not consider possible long-range effects of applying knowledge gained in the research (for example, the possible effects of the research on public policy) as among those research risks that fall within the purview of its responsibility. (CFR §46.111)
“Popularity” is a Ruse
In addition to the ethical fallacy just discussed, there is another fallacy having to do with popularity: the methods in question aren’t very popular. In the medical literature, the typical article about an implausible health claim begins with the irrelevant and erroneous assertion that “34%” or “40%” or even “62%” (if you count prayer!) of Americans use ‘CAM’ each year. This is irrelevant because at issue is the claim in question, not ‘CAM’ in general. It is erroneous because ‘CAM’ in general is so vaguely defined that its imputed popularity has been inflated to the point of absurdity, as exemplified by the NCCAM’s attempt, in 2002, to include prayer (which it quietly dropped from the subsequent, 2007 survey results).
It is erroneous also because it fails to distinguish between such different issues as consulting a practitioner and casually purchasing a vitamin pill at the supermarket, or between Weight Watchers and the pseudoscientific “blood type diet,” and much more. It is erroneous also because it fails to distinguish between occasional and frequent use or between rational use and flimflam (vitamins for deficiency states vs. vitamins to shrink tumors; visualization for anxiety vs. visualization to shrink tumors).
Most of the ‘CAM’ claims for which people consult practitioners are fringe methods, each involving, in the most credible survey, less than 1% of the adult population. The slightly more popular exceptions are chiropractic and massage, reported by 3.3% and 2%, respectively, but these numbers also fail to distinguish rational expectations from flimflam (a wish to alleviate back pain or muscle soreness vs. a wish to cure asthma or to remove ‘toxins’). The most recent National Health Interview Survey (NHIS), co-authored by an NCCAM functionary, reported that 8.6% of adults had used “chiropractic or osteopathic manipulation” in the previous 12 months, further confusing the question of chiropractic.
Let’s revisit an example of how the ‘popularity’ gambit has been used to entice scientific reviewers and taxpayers to pony up for regrettable ‘CAM’ research. The aforementioned NCCAM/NHLBI-sponsored Trial to Assess Chelation Therapy for coronary artery disease (TACT), which at $30 million and nearly 2400 subjects was to be the most expensive and largest NIH-sponsored ‘CAM’ trial when it began in 2003, was heralded as follows:
“The public health imperative to undertake a definitive study of chelation therapy is clear. The widespread use of chelation therapy in lieu of established therapies, the lack of adequate prior research to verify its safety and effectiveness, and the overall impact of coronary artery disease convinced NIH that the time is right to launch this rigorous study,” said Stephen E. Straus, M.D., NCCAM Director.
Over 800,000 patient visits were made for chelation therapy in the United States in 1997…
In the application that won him the TACT grant, Dr. Gervasio Lamas, who had also been the author of the American Heart Journal editorial quoted above, used similar language:
2.0 BACKGROUND AND SIGNIFICANCE
2.1 Alternative Medicine and Chelation Therapy in the United States
…A carefully performed national survey, and other more restricted local surveys all find the practice of alternative medicine to be widespread…34% reported using at least one alternative therapy in the last year…Thus alternative medical practices are common, and constitute a significant and generally hidden health care cost for patients.
NCCAM estimated that more than 800,000 visits for chelation therapy were made in the U.S. in 1997…
Sounds impressive, huh? Less so when you know the truth. The NCCAM made no such estimation. It merely accepted, without question, the number given to it by the American College for Advancement in Medicine (ACAM)—a tiny group of quacks who’d been peddling chelation for decades, especially so after their original snake oil-of-choice, Laetrile, had been outlawed. Not mentioned by the NCCAM press release or by Dr. Lamas was that the purported 800,000 chelation visits were for all comers: the ACAM member appointed as TACT “Trial Chelation Consultant” touts chelation for about 70 indications (it’s the One True Cure), so we can only guess how many hapless chelation recipients thought they were being treated for coronary disease.
For the NIH to have chosen ‘visits’ as the units of popularity, moreover, was misleading in itself: each person submitting to chelation typically makes at least 30 biweekly visits followed by indefinite bimonthly visits, so even if the ACAM number had been accurate, about 0.01% of the U.S. adult population underwent the treatment in 1997—a far cry from “34%.”
It’s no surprise, then, that the TACT has dragged on considerably longer than the originally planned 5 years. It hasn’t been able to recruit enough subjects! If you wade through the history of the trial on ClinicalTrials.gov, you’ll find that the expected subject enrollment has dwindled from 2372 to 1700, in spite of the NIH having selected more than 100 “community chelation practices” as study sites, in spite of its having added 22 (originally unplanned) Canadian sites a few years later, and in spite of the trial’s duration having been prolonged by several years.
A brief perusal of the 2002 NHIS data reveals that the NCCAM could have predicted this problem: the survey estimated that 0.0% (sic) of the U.S. adult population had used chelation for any reason in the previous 12 months, based on 10 of 31,000 adults interviewed having answered in the affirmative—a number so small that the extrapolation to the entire population “did not meet standards of reliability or precision.” Do you suppose that Director Straus was aware of the NHIS data when he asserted a “widespread use of chelation therapy”?

   Of SBM and EBM Redux. Part III: Parapsychology is the Role Model for “CAM” Research
Kimball Atwood
Link
This is the third post in this series*; please see Part II for a review. Part II offered several arguments against the assertion that it is a good idea to perform efficacy trials of medical claims that have been refuted by basic science or by other, pre-trial evidence. This post will add to those arguments, continuing to identify the inadequacies of the tools of Evidence-Based Medicine (EBM) as applied to such claims.
Prof. Simon Replies
Prior to the posting of Part II, statistician Steve Simon, whose views had been the impetus for this series, posted another article on his blog, responding to Part I of this series. He agreed with some of what both Dr. Gorski and I had written:
The blog post by Dr. Atwood points out a critical distinction between “biologically implausible” and “no known mechanism of action” and I must concede this point. There are certain therapies in CAM that take the claim of biological plausibility to an extreme. It’s not as if those therapies are just implausible. It is that those therapies must posit a mechanism that “would necessarily violate scientific principles that rest on far more solid ground than any number of equivocal, bias-and-error-prone clinical trials could hope to overturn.” Examples of such therapies are homeopathy, energy medicine, chiropractic subluxations, craniosacral rhythms, and coffee enemas.
The Science Based Medicine site would argue that randomized trials for these therapies are never justified. And it bothers Dr. Atwood when a systematic review from the Cochrane Collaboration states that no conclusions can be drawn about homeopathy as a treatment for asthma because of a lack of evidence from well conducted clinical trials. There’s plenty of evidence from basic physics and chemistry that can allow you to draw strong conclusions about whether homeopathy is an effective treatment for asthma. So the Cochrane Collaboration is ignoring this evidence, and worse still, is implicitly (and sometimes explicitly) calling for more research in this area.
On the other hand:
There are a host of issues worth discussing here, but let me limit myself for now to one very basic issue. Is any research justified for a therapy like homeopathy when basic physics and chemistry will provide more than enough evidence by itself to suggest that such research is futile(?) Worse still, the randomized trial is subject to numerous biases that can lead to erroneous conclusions. 
I disagree for a variety of reasons.
Prof. Simon offered 5 reasons, quoted here in part:
     1.  It’s good for business. I don’t want to sound shallow, but there’s money to be made by statisticians when research is done, and if I make a few bucks and in the process help to make the research more rigorous, that’s a win-win situation. I’ve not done much work with CAM, but I have helped on several projects at Cleveland Chiropractic College…
     2.  Everyone deserves their day in court. I believe that if someone is sincere in testing whether a therapy is effective or not, then they deserve my help…
     3.  CAM therapies represent an enormous expenditure of limited health care dollars, and if research can help limit the fraction of CAM expenditures that are inappropriate then that represents a good use of scarce research dollars…
     4.  We have to trust that the system can work. Randomized trials are indeed subject to many biases, and it is worth noting them. But are the biases so serious that they will lead to incorrect conclusions about CAM? [etc.]
     5.  Scientific testing is the norm for other claims that lack scientific plausibility. I am a regular (reader) of Skeptical Inquirer and Skeptic Magazine, and when someone makes a claim about ghosts, telekinesis, or reincarnation, they’ll point out all the existing knowledge that makes such claims unbelievable. But then they’ll still go to the haunted house or set up a spoon bending experiment or reinterview people who remember past lives. These claims have even less credibility than much of CAM research, but they are still being tested. So why not test CAM the same way?
I take the “I don’t want to sound shallow” remark at face value, although I’d remind Prof. Simon that without the ‘subluxation,’ whose fatal implausibility he appears to have conceded, chiropractic is left with very little. Reasons 1, 2, and 4 amount to the same assertions: that efficacy trials (“testing whether a therapy is effective or not”) of futile methods have something worthwhile to add to the already compelling evidence against those methods; that they will be performed safely and ethically; that they will dependably show that the methods are ineffective beyond ‘placebo effects’ (we’ve already agreed on this, no?); and that EBM referees, such as those at Cochrane, will subsequently judge such methods futile.
I’ve previously offered several counterexamples to those assertions, including the Cochrane homeopathy reviews quoted in Part I, and the Cochrane “Touch Therapies” review linked from Part II. I’ve also offered examples of methods that are not quite as implausible but are dangerous and have been sufficiently refuted by other means, including biology and even clinical tests, but that EBM experts have deemed worthy of further testing: Laetrile (discussed in Part I), Na2EDTA “chelation therapy” for atherosclerotic cardiovascular disease, and the “Gonzalez Regimen” for cancer of the pancreas (each discussed in Part II).
Can “Research Help Limit CAM Expenditures”?
I’ll discuss such issues more below, including a response to Prof. Simon’s point 5, but first let me briefly address his point 3. The assertion that there is societal value to studying implausible methods has been the usual justification for such trials, as I discussed at some length in Part II. It began as an untested presumption in itself, and it does not excuse endangering experimental subjects or siphoning scarce public funds away from promising research. Moreover it would be, at best, redundant: if other facts are sufficient to refute a claim, there is no point in subjecting the claim to a trial. If people don’t understand that point, then it is the job of experts to explain it to them, not to devalue science by granting every preposterous notion a “day in court” that it has already had, or by issuing preposterous opinions such as “it is not possible to comment on the use of homeopathy in treating dementia.”
Regarding the presumption that “research can help limit the fraction of CAM expenditures that are inappropriate,” the evidence, such as it is, suggests otherwise. In the 1980s, Petr Skrabanek could accurately report that “numerous controlled trials have shown that acupuncture is nothing more than a placebo.” Yet even as additional, abundant, increasingly rigorous trials have relentlessly shown the same thing, acupuncture has steadily increased in popularity. The same is true for homeopathy.
Referring to slightly more plausible methods, Josephine Briggs, the Director of the NCCAM, reported that sales of echinacea, glucosamine-chondroitin sulfate, and gingko biloba had declined after disconfirming trials funded by her Center, but according to Steve Novella the decline was only temporary for echinacea. Perhaps some industrious reader can find data for the other two preparations—I don’t feel like shelling out $200+.
Such Trials Don’t Work
The final reason that efficacy trials of highly implausible claims are a bad idea is that they don’t work very well: they tend to yield, in the aggregate, equivocal, rather than merely disconfirming results. Yes, the biases are so serious that they have led to incorrect conclusions about CAM, at least for a substantial period. This is something that most physicians and even many statisticians seem unaware of, although it was utterly predictable. I’ve discussed this at length, beginning here:
EBM and “CAM”
To many in this era of EBM it seems self-evident that all unproven methods, including homeopathy, should be subjected to such scrutiny. After all, the anecdotal impressions that are typically the bases for such claims are laden with the very biases that blinded RCTs were devised to overcome. This opinion, however, is naive. Some claims are so implausible that clinical trials tend to confuse, rather than clarify the issue. Human trials are messy. It is impossible to make them rigorous in ways that are comparable to laboratory experiments. Compared to laboratory investigations, clinical trials are necessarily less powered and more prone to numerous other sources of error: biases, whether conscious or not, causing or resulting from non-comparable experimental and control groups, cuing of subjects, post-hoc analyses, multiple testing artifacts, unrecognized confounding of data due to subjects’ own motivations, non-publication of results, inappropriate statistical analyses, conclusions that don’t follow from the data, inappropriate pooling of non-significant data from several, small studies to produce an aggregate that appears statistically significant, fraud, and more.
Most of those problems are not apparent in primary reports. Several have already been discussed or referenced elsewhere on this site: here, here, here and here, for example. Academics active in the EBM movement are aware of most of them and want to correct them—as a quick scan of the contents of almost any major medical journal will reveal.
It is clear that such biases are more likely to skew the results of studies that are funded or performed by advocates. This has been found in studies of trials funded by drug companies, for example, as referenced here. In the case of “CAM,” the charge is supported by the preponderance of favorable reports in advocacy journals (here, here, and here) and by examples of overwhelmingly favorable reports emanating from regions with strong political motivations.
For those reasons we can predict that RCTs of ineffective claims championed by impassioned advocates will demonstrate several characteristics. Small studies, those performed by advocates or reported in advocacy journals, and those judged to be of poor quality will tend to be “positive.” The larger the study and the better the design, the more likely it is to be “negative.” Over time, early “positive” trials and reviews will give way to negative ones, at least among those judged to be of high quality and reported in reputable journals. In the aggregate, trials of ineffective claims championed by impassioned advocates will appear to yield equivocal rather than merely “negative” outcomes. The inevitable, continual citations of dubious reports will lead some to judge that the aggregate data are “weakly positive” or that the treatment is “better than placebo.” An example is the claim that stimulation of the “pericardium 6” acupuncture point is effective in the prevention and treatment of post-operative nausea and vomiting—a purportedly proven “CAM” method.
Homeopathic “Remedies” are Placebos
After 200 years and numerous studies, including many randomized, controlled trials (RCTs) and several meta-analyses and systematic reviews, homeopathy has performed exactly as described above. The best that proponents can offer is equivocal evidence of a weak effect compared to placebo. That is exactly what is expected if homeopathy is placebo.
Nevertheless, EBM advocates on the whole don’t see it that way. Those who want to see homeopathy vindicated, such as homeopath Wayne Jonas, the former director of the NIH Office of Alternative Medicine, point to the weakly positive evidence. Others, even those who find homeopathy implausible, are so convinced that EBM can answer the question (“Either homeopathy works or controlled trials don’t!”) that they call for more trials, with no end in sight. Such judgments expose a major weakness in EBM that is not apparent when the exercise is applied to plausible claims.
“CAM” Research and Parapsychology
That passage was a prelude to introducing the EBM “Levels of Evidence” scheme and the Cochrane abstracts later discussed in Part I of this series. It applies equally well to acupuncture, “energy medicine,” and other highly implausible claims that have been subjected to efficacy trials.
Here we come to Prof. Simon’s point 5, that “scientific testing is the norm for other claims that lack scientific plausibility,” such as ghosts and telekinesis. It is true that “psychic detectives” such as Randi and Joe Nickell and Ray Hyman and Richard Wiseman have tested such claims and continue to do so, but Prof. Simon ought to understand the differences between such tests and what’s at issue here. The former tend to be of the sort that I favored in Part II: simple (bias-resistant), inexpensive, performed by skeptics, with the onus of proof placed on the claimants. Such testing, moreover, is fun, which I believe is the main reason that Randi and others are drawn to it.
Typical “CAM” efficacy trials are altogether different, as I began to explain in Part II: they are expensive, messy, and bias-prone, and those who perform them are often enthusiasts or otherwise credulous. Such trials are akin to tests of telekinesis performed not by Randi or Wiseman, but by hopeful or true-believing parapsychologists—and that is exactly what is reflected in EBM-style reviews of their outcomes. Ironically, some “CAM” efficacy trials really are tests of telekinesis performed by true-believing parapsychologists, and much of “CAM” is nothing more than recycled psi claims now pitched to a naïve audience, as discussed here under “The Psi Myth.”
Thus homeopath David Reilly was correct, as I wrote here, when he asserted that “either homeopathy works or controlled trials don’t”:
…but not in the way that he supposed. If there is anything that the history of parapsychology can teach the biomedical world, it is the point just made: that human RCTs, as good as they are at minimizing bias or chance deviations from population parameters, cannot ever be expected to provide, by themselves, objective measures of truth. There is still ample room for erroneous conclusions. Without using broader knowledge (science) to guide our thinking, we will plunge headlong into a thicket of errors—exactly as happened in parapsychology for decades and is now being repeated by its offspring, “CAM” research.
Yes, “CAM” has much to owe to parapsychology, none of it good. Prof. Simon, a statistician, might consider that parapsychology has flirted with the barely positive side of the null effect for decades. Its apparent successes, modest and irreproducible though they’ve been, have rendered it an immortal field of fruitless inquiry: a pathological science.
John Ioannidis has this to say about such a field:
History of science teaches us that scientific endeavor has often in the past wasted effort in fields with absolutely no yield of true scientific information, at least based on our current understanding. In such a “null field,” one would ideally expect all observed effect sizes to vary by chance around the null in the absence of bias. The extent that observed findings deviate from what is expected by chance alone would be simply a pure measure of the prevailing bias.
EBM, Eventually, Sort of Works
For fairness’ sake, let me mention that two veteran “CAM” researchers, Edzard Ernst and R. Barker Bausell (a statistician), eventually decided that most of what they had studied was bogus, and they seem to have arrived at this realization after examining the results of efficacy trials. Thus it is true that EBM, as it is currently practiced, can lead some researchers to rational conclusions.
The problem is far from solved, however: in the cases of the two researchers just mentioned, it took years for them to arrive at a truth that was always staring them in the face (as I discussed in Part I and elsewhere), during which time the waste, the unethical treatment of human subjects, and the false promise that is “CAM” research marched on. Because of the continued, inevitable, equivocal results of such research, moreover, the views of Ernst and Bausell are not shared by other “CAM” research enthusiasts, and probably won’t be anytime soon.
EBM Ignores External Evidence, but not Entirely: a Prelude to Part IV
To reiterate, the major problem with EBM, as it has been applied to implausible medical claims, is that it fails to give adequate strength to evidence from sources other than RCTs. Yet RCTs involving numerous experimental variables and outcomes, as have been typical for “CAM” efficacy trials, are prone to numerous errors and biases, whereas other sources of evidence can be definitive—as has been the case for homeopathy, Laetrile, Therapeutic Touch, chelation for atherosclerosis, and Craniosacral Therapy, for example.
For the first time in several years, motivated by this series, I’ve looked at a few complete Cochrane “CAM” Reviews. In the final posting I’ll discuss more external evidence that is missing from those reviews, but I’ll also report a couple of pleasant surprises. It turns out that Steve Simon was not entirely wrong when he asserted that “people within EBM (are) working both formally and informally to replace the rigid hierarchy with something that places each research study in context.” They have a long way to go, but there is at least a suggestion of change in that direction.
I’ll also address, I hope briefly, this statement by Prof. Simon:
Also how can we invoke scientific plausibility in a world where intelligent people differ strongly on what is plausible and what is not? Finally, is there a legitimate Bayesian way to incorporate information about scientific plausibility into a Cochrane Collaboration systematic overview(?)
Laydah.

   Of SBM and EBM Redux. Part IV: More Cochrane and a little Bayes
Kimball Atwood
Link
Review
This is the fourth and final part of a series-within-a-series* inspired by statistician Steve Simon. Professor Simon had challenged the view, held by several bloggers here at SBM, that Evidence-Based Medicine (EBM) has been mostly inadequate to the task of reaching definitive conclusions about highly implausible medical claims. In Part I, I reiterated a fundamental problem with EBM, reflected in its Levels of Evidence scheme, that although it correctly recognizes basic science and other pre-clinical evidence as insufficient bases for introducing novel treatments into practice, it fails to acknowledge that they are necessary bases. I explained the difference between “plausibility” and “knowing the mechanism.”
I showed, with several examples, that in the EBM lexicon the word “evidence” refers almost exclusively to the results of clinical trials: thus, when faced with equivocal or no clinical trials of some highly implausible claim, EBM practitioners typically declare that there is “not enough evidence” to either accept or reject the claim, and call for more trials—although in many cases there is abundant evidence, other than clinical trials, that conclusively refutes the claim. I rejected Prof. Simon’s assertion that we at SBM want to “give (EBM) a new label,” making the point that we only want it to live up to its current label by considering all the evidence. I doubted Prof. Simon’s contention that “people within EBM (are) working both formally and informally to replace the rigid hierarchy with something that places each research study in context.”
In Part II I responded to the widely held assertion, also held by Prof. Simon, that there is “societal value in testing (highly implausible) therapies that are in wide use.” I made it clear that I don’t oppose simple tests of basic claims, such as the Emily Rosa experiment, but I noted that EBM reviewers, including those employed by the Cochrane Collaboration, typically ignore such tests. I wrote that I oppose large efficacy trials and public funding of such trials. I argued that the popularity gambit has resulted in human subjects being exposed to dangerous and unethical trials, and I quoted language from ethics treatises specifically contradicting the assertion that popularity justifies such trials. Finally, I showed that the alleged popularity of most “CAM” methods—as irrelevant as it may be to the question of human studies ethics—has been greatly exaggerated.
In Part III I continued to argue against trials of implausible methods. I didn’t share Prof. Simon’s optimism, expressed in another post on his blog, that “research can help limit the fraction of CAM expenditures that are inappropriate.” I argued that whatever evidence there is suggests otherwise. I argued that if existing science is sufficient to reject a method, as is the case for much of “CAM,” then the research has already been done, and the task of EBM is to explain this kind of evidence—not to pretend that the jury is still out. I argued, furthermore, that efficacy trials of highly implausible, ineffective methods inevitably yield equivocal, rather than merely disconfirming results, and that this leads to an endless cycle of further (equivocal) trials. I offered parapsychology as a longstanding example of such an “an immortal field of fruitless inquiry: a pathological science.”
I promised that in this final part of the series I would mention a few more points about Cochrane Reviews of highly implausible methods, and even report some reasons for slight optimism. Finally, I promised to respond briefly to this comment by Prof. Simon:
…how can we invoke scientific plausibility in a world where intelligent people differ strongly on what is plausible and what is not? Finally, is there a legitimate Bayesian way to incorporate information about scientific plausibility into a Cochrane Collaboration systematic overview(?)
Cochrane Reviews
The ongoing theme is that Cochrane Reviews ignore key ‘external’ evidence, by which I mean all evidence other than what might be found in randomized, controlled trials (RCTs). I’ve previously alluded to the 2002 Cochrane Review of “Chelation Therapy for Atherosclerotic Cardiovascular Disease,” which concluded,
At present, there is insufficient evidence to decide on the effectiveness or ineffectiveness of chelation therapy in improving clinical outcomes of people with atherosclerotic cardiovascular disease.
Elsewhere we have shown that the evidence against such effectiveness is substantial, far exceeding the evidence against Laetrile for cancer or against bilateral ligation of the internal mammary arteries for coronary disease, two long-since discredited methods that no biomedical researcher in his right mind would consider resurrecting for further trials. Oops, make that almost no biomedical researcher.
Quips aside, my reason for bringing up the chelation review again is as follows. The review acknowledges that there have been RCTs, involving about 250 subjects, which
…showed no significant difference in the following outcomes: direct or indirect measurement of disease severity and subjective measures of improvement.
That those findings weren’t sufficient reasons for the Cochrane reviewers to judge chelation ineffective was apparently due first to their having ignored the abundant non-RCT evidence, and second to their having been intrigued by a single, tiny RCT that reported a positive outcome:
One of the studies, which included only 10 participants, was interrupted prematurely, because of an apparent treatment effect. However, relevant data were not available in the report and have been requested from the authors.
I had to chuckle when I read that passage, because I know a lot about that study and its authors. If you look here and scroll down to “Olszewer (1988),” you will know, too. I wonder if the Cochrane Reviewers ever got a straight story from the authors. There hasn’t been an update of that review, so whether they did or not is anyone’s guess.
I’m sorry to say that I haven’t been able to get a copy of that review in its entirety, so my comments apply only to the abstract. I have recently obtained a few other complete reviews, of which two are worth mentioning. A 2008 review of “Touch therapies for pain relief in adults” looked at Healing Touch, Therapeutic Touch, and Reiki. It doesn’t mention the Emily Rosa experiment. Its conclusion is what we’ve come to expect:
Touch therapies may have a modest effect in pain relief. More studies on HT and Reiki in relieving pain are needed. More studies including children are also required to evaluate the effect of touch on children.
In this review we are told why it is that not touching is called “touch”:
Touch Therapies are so-called as it is believed that the practitioners have touched the clients’ energy ﬁeld.
For readers who are unfamiliar with such practices, which consist of waving one’s hands over a “client” (you really have to see it to believe it), the review continues:
It is believed this effect occurs by exerting energy to restore, energize, and balance the energy ﬁeld disturbances using hands-on or hands-off techniques (Eden 1993). The underlying concept is that sickness and disease arise from imbalances in the vital energy ﬁeld. However, the existence of the energy ﬁeld of the human body has not been proven scientiﬁcally and thus the effect of such therapies, which are believed to exert an effect on one’s energy ﬁeld, is controversial and lies in doubt.
Indeed. The following passages are not to be read in detail, other than by the masochistic. Their purpose is to demonstrate the elaborate wheel spinning that EBM treatments of such fanciful methods inevitably involve. The perseveration of statistics, as if they can support the house of cards that is the basis for the technique, will, I hope, ruffle Prof. Simon’s own energy field:
Types of touch therapies
The effects of different kinds of Touch Therapies were examined (Comparison 3). It appears that all three types of Touch Therapy, HT, TT and Reiki, may decrease pain to a certain extent. Substantial heterogeneity exists among the HT group and TT group. The HT group (163 participants) had an I2 of 76% and a P value= 0.04 (Chi-square) and the TT group (686 participants) had an I2 of 70% and a P value < 0.00001 (Chi-square). The results for both the TT and the HT group indicate that there is signiﬁcant heterogeneity and that the effects were positive. There were two studies in HT included in the analyses (Cook 2004; Post-White 2003). The pooled results showed that participants exposed to HT had, on average, 0.71 units less pain, however, this was not statistically signiﬁcant (95% CI: -2.27 to 0.86). The pooled estimates of TT suggested a statistically signiﬁcant result of 0.81 units (95% CI: -1.19 to -0.43) less pain in the exposed group. Nonsigniﬁcant heterogeneity was detected in the Reiki group (116 participants) which consisted of three studies (Dressen 1998; Olson 2003; Tsang 2007) with an I2 of 7% and a P value = 0.34 (Chi-square). The pooled estimates of the results for Reiki reported a statistically signiﬁcant effect. Participants exposed to Reiki had an average of 1.24 (95% CI: -2.06 to -0.42) less pain.
Experience of practitioner
The experience of the practitioner on the effects of touch therapies was also analyzed. This helped to explore whether a less experienced touch practitioner would result in less effect and an experienced touch practitioner would result in an increased effect. Subgroup analyses were thus performed. There are usually four levels of training in HT, TT and Reiki, level I, II, III and a master level or teacher level. Studies were divided into subgroups according to the level of training. Due to a small number of studies having reported the experience of touch practitioners, studies were divided into two groups, level I and II , and level III or above, rather than four groups. Four studies were included in the subgroup of less experienced practitioners (212 participants) (Blankﬁeld 2001; Cook 2004; Frank 2003; Redner 1991). An I2 of 2% and a P value = 0.38 (Chi-square) was found. In the subgroup of more experienced practitioners (116 participants), three studies were included (Dressen 1998; Olson 2003; Tsang 2007) and an I2 of 7% and a P value = 0.34 (Chi-square) was found. No signiﬁcant heterogeneity was detected in the two subgroups. Participants in the subgroup of less experienced practitioners had, on average, 0.47 units (95% CI: -0.73 to -0.22) less pain. More experienced practitioners yielded higher contribution, having on average of 1.24 units reduction in pain intensity (95% CI: -2.63 to -0.23) (Comparison 4). Minor non-signiﬁcant heterogeneity existed in both the experienced and less-experienced group. This might suggest that any heterogeneity calculated in other subgroups was owing to the difference in the experience of the practitioners. However, only a small number of identiﬁed studies were included in this subgroup and the apparent small heterogeneity may be owing to the low power of the chi-square test or due to having a small numbers of studies (Higgins 2005), it would not be appropriate to make conclusions about the existence of heterogeneity at this stage with the current results.
Dose-response analyses
Dose-response analyses were also conducted to investigate if there was any difference in effect due to differences in duration of treatment. The study with the shortest session had the session lasting for ﬁve minutes while studies with the longest session lasted for ninety minutes. The number of treatment sessions ranged from a single session to ten sessions. Data regarding the duration and number of treatment sessions were pooled. Data were analyzed in terms of total duration of treatment (duration of a single session multiplied by the number of treatment sessions). Three hundred and ninety six participants exposed to touch for less than an hour had an average of 1.16 units (95% CI: -1.85 to -0.47) less pain; 239 participants exposed to touch for more than one hour but less than two hours had an average of 0.75 units (95% CI: -1.81 to 0.31) less pain, but this was insigniﬁcant; 255 participants exposed to touch for between two to three hours had an average of 0.47 units (95% CI: -1.09, 0.14) less pain; 116 participants exposed for over three hours to touch therapies had an average of 1.57 units less pain (95% CI: -2.38 to -0.76). No dose-response relationship can be gained as yet from this information (Comparison 7).
Phew! (‘Dose-response analyses’ ?). At the end of the monograph are the first author’s acknowledgements:
I would like to thank Dr Yan-Kit Cheung, who is my Reiki teacher and an expert in complementary and alternative medicine, in giving me valuable advice in conducting this review…Last but not least, I would like to express my gratitude to Miss Wan-Choi Patsy Lee who brought me to Reiki. Without her, I would know nothing about Reiki and would not be enlightened by this precious gift.
I don’t know who should be more embarrassed: Cochrane, for having asked a devout believer to pass judgment on the sacred object of her belief, or me, for having stumbled upon her supplication.

   Of SBM and EBM Redux. Part IV, Continued: More Cochrane and a little Bayes
Kimball Atwood 
Link
OK, I admit that I pulled a fast one. I never finished the last post as promised, so here it is.
Cochrane Continued
In the last post I alluded to the 2006 Cochrane Laetrile review, the conclusion of which was:
This systematic review has clearly identified the need for randomised or controlled clinical trials assessing the effectiveness of Laetrile or amygdalin for cancer treatment.
I’d previously asserted that this conclusion “stand[s] the rationale for RCTs on its head,” because a rigorous, disconfirming case series had long ago put the matter to rest. Later I reported that Edzard Ernst, one of the Cochrane authors, had changed his mind, writing, “Would I argue for more Laetrile studies? NO.” That in itself is a reason for optimism, but Dr. Ernst is such an exception among “CAM” researchers that it almost seemed not to count.
Until recently, however, I’d only seen the abstract of the Cochrane Laetrile review. Now I’ve read the entire review, and there’s a very pleasant surprise in it (Professor Simon, take notice). In a section labeled “Feedback” is this letter from another Cochrane reviewer, which was apparently added in August of 2006, well before I voiced my own objections:
The authors’ state that they: “[have] clearly identified the need for randomised or controlled clinical trials assessing the effectiveness of Laetrile or amygdalin for cancer treatment.” This is to fail completely to understand the nature of oncology research in which agents are tested in randomized trials (“Phase III”) only after they have been successful in Phase I and II study. There was a large Phase II study of laetrile (N Engl J Med. 1982 Jan 28;306(4):201-6) which the authors of the review do not cite, they merely exclude as being non-randomized. But the results of the paper are quite clear: there was no evidence that laetrile had any effect on cancer (all patients had progression of disease within a few months); moreover, toxicity was reported. To expose patients to a toxic agent that did not show promising results in a single arm study is clinical, scientific and ethical nonsense.
I would like to make a serious recommendation to the Cochrane Cancer group that no reviews on cancer are published unless at least one of the authors either has a clinical practice that focuses on cancer or actively conducts primary research on cancer. My recollection when the Cochrane collaboration was established was that the combination of “methodologic” and “content” expertise was essential.
Wow! That letter makes several of the same arguments that we’ve made here: that for both scientific and ethical reasons, scientific promise (including success in earlier trials) ought to be a necessary pre-requisite for a large RCT; that the 1982 Moertel case series was sufficient to disqualify Laetrile; and that EBM, at least in this Cochrane review, suffers from “methodolatry.” It also brings to mind Steven Goodman’s words:
An important problem exists in the interpretation of modern medical research data: Biological understanding and previous research play little formal role in the interpretation of quantitative results. This phenomenon is manifest in the discussion sections of research articles and ultimately can affect the reliability of conclusions. The standard statistical approach has created this situation by promoting the illusion that conclusions can be produced with certain “error rates,” without consideration of information from outside the experiment.
…
This method thus facilitated a subtle change in the balance of medical authority from those with knowledge of the biological basis of medicine toward those with knowledge of quantitative methods, or toward the quantitative results alone, as though the numbers somehow spoke for themselves.
Perhaps most surprising about the ‘Feedback’ letter is the identity of its author: Andrew Vickers, a biostatistician who wrote the Center for Evidence-Based Medicine’s “Introduction to evidence-based complementary medicine.” I’ve complained about that treatise before in this long series, observing that
There is not a mention of established knowledge in it, although there are references to several claims, including homeopathy, that are refuted by things that we already know.
Well, Dr. Vickers may not have considered plausibility when he wrote his Intro to EBCM, but he certainly seems to have done so when he wrote his objection to the Cochrane Laetrile review. Which is an appropriate segue to a topic that Dr. Vickers hints at (“content expertise”), perhaps unintentionally, in the letter quoted above: Bayesian inference.
Bayes Revisited
A few years ago I posted three essays about Bayesian inference: they are linked below (nos. 2-4). The salient points are these:
    • Bayes’s Theorem is the solution to the problem of inductive inference, which is how medical research (and most science) proceeds: we want to know the probability of our hypothesis being true given the data generated by the experiment in question.
    • Frequentist inference, which is typically used for medical research, applies to deductive reasoning: it tells us the probability of a set of data given the truth of a hypothesis. To use it to judge the probability of the truth of that hypothesis given a set of data is illogical: the fallacy of the transposed conditional.
    • Frequentist inference, furthermore, is based on assumptions that defy reality: that there have been an infinite number of identically designed, randomized experiments (or other sort of random sampling), without error or bias.
    • Bayes’s Theorem formally incorporates, in its “prior probability” term, information other than the results of the experiment. This is the sticking point for many in the EBM crowd: they consider prior probability estimates, which are at least partially subjective, to be arbitrary, capricious, untrustworthy, and—paradoxically, because it is science that is ignored in the breach—unscientific.
    • Nevertheless, prior probability matters whether we like it or not, and whether we can estimate it with any certainty or not. If the prior probability is high, even modest experimental evidence supporting a new hypothesis deserves to be taken seriously; if it is low, the experimental evidence must be correspondingly robust to warrant taking the hypothesis seriously. If the prior probability is infinitesimal, the experimental evidence must approach infinity to warrant taking the hypothesis seriously.
    • Frequentist methods lack a formal measure of prior probability, which contributes to the seductive but erroneous belief that “conclusions can be produced…without consideration of information from outside the experiment.”
    • The Bayes Factor is a term in the theorem that is based entirely on data, and is thus an objective measure of experimental evidence. Bayes factors, in the words of Dr. Goodman, “show that P values greatly overstate the evidence against the null hypothesis.”
I bring up Bayes again to respond to Prof. Simon’s statements, recently echoed by several readers, that people may differ strongly in what they consider plausible, and that it is not clear how prior probability estimates might be incorporated into formal reviews. I’ve discussed these issues previously (here and here, and in recent comments here and here), but it is worth adding a point or two.
First, it doesn’t really matter that people may differ strongly in what they consider plausible. What matters is that they commit to some range of plausibility—in public and with justifications, in the cases of authors and reviewers, so that readers will know where they stand—and that everyone understands that this matters when it comes to judging the experimental evidence for or against a hypothesis.
An example will explain these points. Wayne Jonas was the Director of the US Office of Alternative Medicine from 1995 until its metamorphosis into the NCCAM in 1999. He is the co-author, along with Jennifer Jacobs, of Healing with Homeopathy: the Doctors’ Guide (©1996), which unambiguously asserts that ultra-dilute homeopathic preparations have specific effects. Yet Jonas is also the co-author (with Klaus Linde) of a 2005 letter to the Lancet that includes this statement, prefacing his argument that homeopathy, already subjected to hundreds of clinical trials, has not been disproved and deserves further trials:
We agree that homoeopathy is highly implausible and that the evidence from placebo-controlled trials is not robust.
Bayes’s theorem shows that Jonas can’t have it both ways. Either he doesn’t really agree that homeopathy is highly implausible (which seems likely, unless he changed his mind between 1996 and 2005—oops, he didn’t); or, if he does, he needs to recognize that his statement quoted above is equivalent to arguing that the homeopathy ‘hypothesis’ has been disproved, at least to an extent sufficient to discourage further trials.
Next, does it matter that we can’t translate qualitative statements of plausibility to precise quantitative measures? Does this mean that prior probability, in the Bayesian sense, is not applicable? I don’t think so, and neither do many scientists and statisticians. Even “neutral” or “non-informative” priors, when combined with Bayes factors, are more useful than P values (see #7 above). “Informative” priors—estimated priors or ranges of priors based on existing knowledge—are both useful and revealing: useful because they show how differing priors affect the extent to which we ought to revise our view of a hypothesis in the face of new experimental evidence (see #5 above); and revealing of where authors and others really stand, and of the information that those authors have used to make their estimates.
I believe that frequentist statistics has allowed Dr. Jonas and other “CAM” enthusiasts to project a posture of scientific skepticism, as illustrated by Jonas’s words quoted above, without having to accept the consequences thereof. If convention had compelled him to offer a prior high enough to warrant further trials of homeopathy, Dr. Jonas would have revealed himself as credulous and foolish.
Finally, there is no reason that qualitative priors can’t be translated, if not precisely then at least usefully, to estimated quantitative priors. Sander Greenland, an epidemiologist and a Bayesian, explains this in regard to household wiring as a possible risk factor for childhood leukemia. First, he argues that there are often empirical bases for estimating priors:
…assuming (an) absence of prior information is empirically absurd. Prior information of zero implies that a relative risk of (say) 10100 is as plausible as a value of 1 or 2. Suppose the relative risk was truly 10100; then every child exposed >3 mG would have contracted leukaemia, making exposure a sufficient cause. The resulting epidemic would have come to everyone’s attention long before the above study was done because the leukaemia rate would have reached the prevalence of high exposure, or ~5/100 annually in the US, as opposed to the actual value of 4 per 100,000 annually; the same could be said of any relative risk >100. Thus there are ample background data to rule out such extreme relative risks.
The same could be said for many “CAM” methods that, while not strictly subjects of epidemiology per se, have generated ample experimental data (see homeopathy) or have been in use by enough people for enough time to have been noticed for substantial deviations from typical outcomes of universal diseases, should such deviations exist (see “Traditional [insert ethnic group here] Medicine”).
Next, Greenland has no problem with non-empirically generated priors, because these are revealing as well:
Many authors have expressed extreme scepticism over the existence of an actual magnetic-field effect, so much so that they have misinterpreted positive findings as null because they were not ‘statistically significant’ (e.g. UKCCS, 1999). The Bayesian framework allows this sort of prejudice to be displayed explicitly in the prior, rather than forcing it into misinterpretation of the data.
By “misinterpretation,” Greenland is arguing not that the “positive findings” of epidemiologic studies have proven the existence of a magnetic field effect, but that the objections of extreme skeptics must be made explicit: it is their presumed, if unstated, prior probability estimates that justify their conclusions about whether or not there is an actual magnetic field effect associated with childhood leukemia; it is not the data collection itself. Prior probability estimates put people’s cards on the table.
I recommend the rest of Greenland’s article, which is full of interesting stuff. For example, he doesn’t agree that “objective” Bayesian methods, using non-informative priors (see my point #7 above) are more useful than frequentist methods, since they are really doing the same thing:
…frequentist results are what one gets from the Bayesian calculation when the prior information is made negligibly small relative to the data information. In this sense, frequentist results are just extreme Bayesian results, ones in which the prior information is zero, asserting that absolutely nothing is known about the [question] outside of the study. Some promote such priors as ‘letting the data speak for themselves’. In reality, the data say nothing by themselves: The frequentist results are computed using probability models that assume complete absence of bias and so filter the data through false assumptions.
All for now. In the next post I’ll discuss another Cochrane review that has some pleasant surprises.

The hijacking of evidence-based medicine
A hero of the blog, John Ioannidis, worries that evidence-based medicine has been hijacked, and when Ioannidis says something we at SBM listen. But has EBM been “hijacked”?
David Gorski on March 21, 2016
Link


It’s no secret that I’m a fan of John Ioannidis. So, I daresay, are pretty much all of the editors and regular contributors to this blog. (If you don’t believe me, just type Ioannidis’ name into the blog search box and see how many posts you find.) Over the last couple of decades, Ioannidis has arguably done more to reveal the shortcomings of the medical research enterprise that undergirds our treatments, revealing the weaknesses in the evidence base and how easily clinical trials can mislead, than any other researcher. Indeed, after reading what is Ioannidis’ most famous article, “Why Most Published Research Findings Are False“, back in 2005, I was hooked. I even used it for our surgical oncology journal club at the cancer center where I was faculty back then. This was long before I appreciated the difference between science-based medicine (SBM) and evidence-based medicine (EBM). So it was with much interest that I read an article by him published last week and framed as an open letter to David Sackett, the father of evidence-based medicine, entitled “Evidence-based medicine has been hijacked: a report to David Sackett.” Ioannidis is also quoted in a follow-up interview with Retraction Watch.
Before I get to Ioannidis’ latest, I can’t help but point out that, not surprisingly, quacks and proponents of pseudoscientific and unscientific medicine often latch on to Ioannidis’ work to support their quackery and pseudoscience. They’ve been doing it for years. Certainly, they’re already latching on to this article as vindication of their beliefs. After all, their reasoning—if you can call it that—seems to boil down to: If “conventional” medicine is built on such shaky science, then their pseudoscience isn’t wrong after all, given that the same scientific enterprise upon which conventional medicine is based produces the findings that reject their dubious claims and treatments. Of course, whenever I hear this line of argument, I’m reminded of Ben Goldacre’s famous adage, seen in one form on Twitter here:
Quacks citing problems in pharma make me laugh. FLAWS IN AIRCRAFT DESIGN DO NOT PROVE THE EXISTENCE OF MAGIC CARPETS.
— ben goldacre (@bengoldacre) January 31, 2013
The adage can be generalized to all EBM and SBM as well. Just because big pharma misbehaves, EBM has flaws, and conventional medicine practitioners don’t always use the most rigorous evidence does not mean that, for example, homeopathy, acupuncture, or energy medicine works.
Still, when Ioannidis publishes an article with a title provocatively declaring that EBM has been “hijacked,” we at SBM take notice.
Has EBM been “hijacked”?
In his “report” to David Sackett, Ioannidis does touch on a number of pertinent and interesting points regarding the adoption of EBM but, as you will see, pretty much ignores the one huge elephant in the room. Before you hear the pounding of elephant feet, though, let me just say that I think Ioannidis is being a bit too hard on himself to have declared himself a “failure” in 2004 and an “even bigger failure” twelve years later. I suppose by the metrics he uses he is a failure, but I would argue that those are the wrong metrics. First, however, let’s see where Ioannidis is coming from. He believes that EBM met considerable resistance in the US:
EBM met with substantial resistance in the 1990s and 2000s. Even in the USA, the mecca of biomedical research, EBM, and any serious biomedical research that may help intact humans was largely unwanted. As a clinical research fellow, I remember that every week we were waiting to hear whether the Agency for Health Care Policy and Research (AHCPR, which subsequently became AHRQ) would be axed. The agency had hurt the interests of some powerful professional surgical society: one of its guidelines threatened the indications for an expensive and possibly largely useless surgical procedure. AHCPR/AHRQ survived, but has always had to fight valiantly for its existence since then. EBM is widely tolerated mostly when it can produce largely boring evidence reports that are shaped and endorsed by experts. Many years later, the Patient-Centered Outcomes Research Institute was launched, an equally valiant effort to cover some of that vast space. It was also restricted by a mandate not to deal with cost effectiveness of interventions. I was asked to participate in its Methodology Committee along with great colleagues. I contributed far less than anyone else to the effort before deciding to quit out of shame for my lack of contribution. Most of our tasks seemed to require experts rather than evidence. More than 7 billion of people would be better qualified than me to lead expert-based activities.
One can’t help but wonder to which surgical procedure Ioannidis is referring to. Could it be vertebroplasty? Or is Ioannidis referring to the 1990 report by the AHRQ that found rest and pain medications worked as well as surgery for back pain? I think that’s probably it. Be that as it may, it is true that political forces in medicine have long detested the AHRQ. Critics argue that its research is duplicative with the NIH, even though the NIH does not generally fund studies of the health care delivery system as the AHRQ does, and PCORI, even though the PCORI funds only comparative effectiveness research. (Comparative effectiveness research compares the effectiveness of already approved and utilized medical interventions.) Moreover, the agencies all work very hard to avoid duplication.
On the other hand, in organized medicine there is a large constituency that supports the AHRQ, which was in evidence last year during the latest attempt to defund the organization, when dozens of health care organizations raced to the agency’s defense and more than 100 health groups signed a letter in June and July urging the House and Senate to maintain AHRQ and the research it funds, including the American College of Surgeons and the American Association of Orthopedic Surgeons. I also can’t help but note the development of Choosing Wisely four years ago, where physician professional societies consciously agreed to start making lists of interventions that remain common but are not evidence-based in an effort to decrease their use.
So, yes, there is resistance to the AHRQ. However, these days it is far more business interests, such as drug and device manufacturers, than physicians groups who want to abolish both the AHRQ and the PCORI, mainly because the AHRQ and PCORI’s research threatens these companies’ bottom lines by showing which treatments work better in the “real world” and influencing the Centers for Medicare & Medicaid Services (CMS) regarding which new drugs and devices will be paid for. In fact, I’d argue that, while Ioannidis is correct that drug and device manufacturers want to kill AHRQ and PCORI, he’s missed a sea change in attitude among physicians towards such government agencies whose purpose is to evaluate and compare treatments for effectiveness after they’ve been approved. It might have been true that EBM was not popular 15 or 20 years ago, but as new generations of medical students have been inculcated with its principles and importance, EBM has been “baked in” to physician education, with a resultant change in attitude towards efforts to promote EBM. That’s not to say that physician groups don’t protect their turf. Just look at how radiologists, for example, react to new guidelines that increase the recommended age at which to start mammography or how primary care physicians react to legislation expanding the scope of practice of advanced practice nurses. However, extreme hostility to comparative effectiveness research and EBM-based guidelines has mostly retreated to fringe physician groups like the American Association of Physicians and Surgeons. Unfortunately, resistance to EBM as a constraint on physician autonomy is still fairly common, particularly among older physicians.
I view this as one victory of John Ioannidis and others like him. In other words, I agree with much of what he says, but I tend to have a different interpretation, although not always. For instance, it’s very clear that the attacks on the AHRQ and PCORI have little or nothing to do with EBM being “co-opted,” but rather powerful business interests that see EBM-based guidelines promulgated by AHRQ as inimical to their financial interests. Ioannidis even relates a story of a senior professor of cardiologist telling a friend of his that Ioannidis should not be too outspoken, “otherwise Albanian hit men may strangle him in his office.” To me, that’s evidence of success.
Methodolatry and the rumbling of elephants
Here’s where the elephant in the room that I mentioned earlier starts getting closer. Specifically, what Ioannidis complains about next are definitely signs of EBM’s success, but I also agree that they are signs of EBM being co-opted or “hijacked.” After all, if EBM weren’t successful, then why would anyone want to hijack it? The problem is that somehow Ioannidis, as brilliant as he is, doesn’t seem to realize one part of the solution to the problems of EBM that he’s been relating and relates here.
In any case, here’s the next part of Ioannidis’ tale:
Now that EBM and its major tools, randomized trials and meta-analyses, have become highly respected, the EBM movement has been hijacked. Even its proponents suspect that something is wrong [[10], [11]]. The industry runs a large share of the most influential randomized trials. They do them very well, they score better on “quality” checklists [12], and they are more prompt than nonindustry trials to post or publish results [13]. It is just that they often ask the wrong questions with the wrong short-term surrogate outcomes, the wrong analyses, the wrong criteria for success (e.g., large margins for noninferiority), and the wrong inferences [[14], [15], [16]], but who cares about these minor glitches? The industry is also sponsoring a large number of meta-analyses currently [17]. Again, they get their desirable conclusions [18]. In 1999 at the closing session of the Cochrane Colloquium in Rome, among the prevailing enthusiasm of this benevolent community, I spoiled the mirth with my skepticism. I worried that the Cochrane Collaboration may cause harm by giving credibility to biased studies of vested interests through otherwise respected systematic reviews. My good friend, Iain Chalmers, countered that we should not worry—plus there were many topics where the industry had not been involved. He mentioned steroids as one example. It was not very reassuring. Now even the logo of the Collaboration, the forest plot for prenatal steroids, has been shown to be partially wrong due to partial reporting [19]—let alone reviews of trials done with vested interests from their very conception.
It is, of course, not surprising that industry would run the lion’s share of randomized clinical trials (RCTs). The FDA requires RCTs demonstrating efficacy and safety before it will approve a drug or device, and the drug and device industries are in the business of marketing new drugs and devices. It is thus in their financial interest to run the most “rigorous” trials (from an EBM standpoint) that they can and publish them as quickly as they can, because that will speed up FDA approval. Unfortunately, it is also in their interest to ask questions and use endpoints that are most likely to produce “positive” trials. What Ioannidis neglects to mention is that the pressure to do this also comes from non-financial motivations. You have only to look at the activism of AIDS victims in the 1980s and the rise of the “right to try” movement to realize that there is a large and strong constituency demanding faster drug approval. This constituency has influence, too. As a result, legislators are willing introduce legislation that would weaken scientific standards for drug and device approval, like the Saatchi bill in the UK and the 21st Century Cures Act in the US. Not coincidentally, these bills align with industry interests as well as patients looking for faster approval of new drugs. Of course, the answer to this problem is not necessarily a problem with EBM itself. Rather, it is the regulations that drug approval agencies in the US and other countries use to approve drugs and devices.
As for the Cochrane Collaborative, we’ve discussed problems with this group right from the very beginning, most recently when Tom Jefferson of the Cochrane Acute Respiratory Infections Group argued that childhood vaccination schedules are not evidence-based because, in essence, there are not RCTs testing every schedule and RCT evidence refuting various claimed adverse reactions doesn’t exist. This is a problem with EBM that I, in particular, have discussed multiple times before on this blog, the problem of methodolatry, defined as the “profane worship of the randomized controlled clinical trial as the only valid means of investigation.” While it is true that RCTs are the “gold standard” in clinical evidence of efficacy and safety, there are many questions that for either ethical or practical reasons can’t be studied by RCTs. Unfortunately, methodolatry is a feature, not a bug, of EBM that consigns all evidence below that of RCTs and meta-analyses of RCTs to a category of inferior evidence or not evidence at all! That includes the epidemiological studies that have shown vaccines to be safe and not to be a cause of autism, as well as the entire corpus of basic science that tells us homeopathy is impossible pseudoscience. In EBM, it’s RCT evidence and, in particular meta-analyses über alles!
This is why we at SBM frequently refer to the “blind spot” of EBM. From our perspective, EBM has indeed been “hijacked.” I, for example, agree with Ioannidis that industry has to some extent hijacked EBM, but we also add that advocates of quackery have also done so, hence the creation of a whole new field of medicine dedicated to “integrating” pseudoscience with medicine, an NIH center (the National Center for Complementary and Integrative Health), and many programs in academic medical centers dedicated to what I like to call quackademic medicine.
What’s frustrating is that Ioannidis seems at some level to recognize this problem but doesn’t explicitly address it. For instance, he mentions in his article that there are “so many quacks ranging from television presenters and movie stars turned into health trainers [40] and pure science denialists (e.g., climate, HIV, vaccine denialists, and religious fundamentalists) that one has to tread carefully” and that we should “avoid a civil war on how to interpret evidence within the health sciences when so many pseudoscientists and dogmatists are trying to exploit individuals and populations and attack science,” but seems unwilling to address the elephant in the room, how EBM has been hijacked by CAM promoters as well as industry.
Science-Based Medicine: At least part of the cure
We at SBM really like John Ioannidis because we, like he, are dedicated to improving the scientific rigor of medicine. What we have a problem with in EBM is the methodolatry that has come to dominate the core of EBM that leads EBM purists to refuse to dismiss obvious quackery like homeopathy and “energy medicine.” In other words, EBM does not adequately take into account prior probability, and as a result demands RCTs for just about everything. I’m not saying that EBM ignores basic science considerations and the prior probability derived from them; it’s just that EBM relegates basic science to the very lowest rung of the EBM hierarchy. So, on the one hand, EBM (mostly) dismisses basic science considerations, and its methodolatry leads to unethical trials of homeopathy and cancer quackery like the Gonzalez protocol, as well as the widespread acceptance of “integrative medicine” at universities and massive research programs studying quackery (a.k.a. quackademic medicine).
In fact, you can see some of this attitude in Ioannidis’ quoting of Sackett:
We value the study of adenine, thymine, cytosine, and guanine far above the study of childhood diarrhoea, Chagas’ disease, community health, and patients’ decision making. The issue is not the potential usefulness of basic research at some distant future date (although you might even dare question whether it is being oversold). The issue is that basic medical scientists have hijacked the granting bodies and have erected research policies that place greater value in serving their own personal curiosities than in serving sick people.
Perhaps this was true in 2004, although I was around then and am still not sure I buy Sackett’s take even then. It’s certainly not true now, where getting a basic science grant funded by the NIH is damned near impossible without a clear disease or translational research angle. There’s a reason why scientists are calling for more basic science. Whether he realizes it or not, by quoting Sackett, Ioannidis inadvertently appears to be echoing the discounting of basic science at the heart of EBM.
Of course, Ioannidis is not wrong when he observes:
With clinical evidence becoming an industry advertisement tool and with much “basic” science becoming an annex to Las Vegas casinos, how about the other pieces of EBM, for example, diagnosis and prognosis and individualizing care? I have had great excitement about the prospects of omics, big data, personalized medicine, precision medicine, and all. Much of my effort has been to put together these efforts with rigorous statistical methods and EBM tools. But I am tired of seeing the same overrated promises recast again and again. For example, several years ago I gave an invited lecture at a leading institution on the danger of making inflated promises in personalized medicine. Right after my talk, everybody rushed to hear the launch of a new campaign, where the leader of the institution singled out this unique historic moment: that institution would single-handedly eliminate most major types of cancer within a few years. Several years have passed, and none of these cancer types have disappeared. I recently tried to find the name of that campaign online but realized that this institution has launched many similar campaigns. Which among many was the unique historic moment that I happened to be at? Multiply this by thousands of institutions, and there are already millions of unique historic moments where cancer was eliminated. Same applies to neurologic diseases and more. I do not understand why academic leaders and politicians need to make such self-embarrassing announcements now and then.
I do. It’s the same reason why industry has tried to co-opt EBM, and Ioannidis is quite correct to condemn this sort of behavior. Over the years we’ve written many posts criticizing the same sorts of claims. I myself have questioned the hype over “precision medicine,” “liquid biopsies,” and the like. I remember a director of the National Cancer Institute making the claim that we would eliminate the suffering and death from cancer by the year 2015. Well, it’s 2016, and cancer is still with us. I still see a lot of suffering and death due to it. I can still remember attending a surgery meeting and shaking my head as I watched a talk touting this initiative.
So Ioannidis has described much of the disease of EBM, and we’ve tried to describe the rest. What’s the cure?
Our fearless leader Steve Novella argues that SBM is the answer. I agree, but not entirely. To me, SBM is a large part of the answer. Much of the problem with EBM that allows it to be hijacked is EBM’s blind spot with respect to basic science. Steve is correct that SBM is designed to take a “big picture”, global look at any clinical claim and that we need to consider the scientific plausibility of the claim, not just RCTs. However, there is a limit to what SBM can do in correcting the co-optation of EBM.
To understand why, just consider: How do we estimate the basic scientific plausibility of a medical claim? It’s not nearly as straightforward as one might think. How plausible a claim is worth investigating with an RCT? 10% likely to be true? 50% likely to be true? 75% likely to be true? What SBM is great for is rejecting ridiculously implausible health claims, like those made by homeopaths, on the basis of basic science considerations alone that tell us that they are impossible. It’s also great for increasing the rigor of clinical trials by taking scientific mechanism into consideration. It can also be useful for prioritizing which clinical trials to do. I would also argue that SBM can counter the overselling of basic science and early clinical trial results that Ioannidis complains about. Finally, I agree with Ioannidis about how many risk factors for disease have been promoted, based on rather shaky evidence. It’s not for no reason that I thoroughly enjoyed his way of illustrating this point by going through a cookbook to look for which food ingredients have been linked with either increasing or decreasing the list of cancer and finding that most of them have, although the evidence was very shaky.
It has to be noted, however, that, as Ioannidis himself noted, industry clinical trials are often among the most rigorous out there, “checking off all the EBM boxes” for rigor. The same observation also applies to the basic science foundations of these trials. Before a drug company even does its first clinical trial of a new drug, it has reams of basic science data. The mechanism is known and documented. It has to be that way, because the FDA requires it. SBM alone is unlikely to have much of an impact on the hijacking of EBM by industry. Sure, it would help to decrease the number of trials that ask the wrong questions or use the wrong analyses. Similarly, applying prior probability to clinical trial analysis would likely decrease the number of seemingly “positive trials” whose results really aren’t clinically significant. However, I suspect that SBM would be less useful with respect to surrogate outcomes, because choosing surrogate outcomes is a question EBM is already well-equipped to deal with, and, aside from clinical trials of “complementary and alternative medicine” treatments with very low prior probability, the same can be said with regard to the criteria for success and how to derive inferences.
What’s far more likely to have an effect are two things: Transparency, and the requirement of more clinically relevant endpoints by both journal editors and government regulatory agencies. Of the two, transparency is likely to be very important, which is why the AllTrials movement is so important, to make sure that the results of every RCT is recorded somewhere, including the negative ones that all too often are not published. As hard as making sure the results of all clinical trials are reported in a timely fashion is, herding the cats to get them to agree on what constitutes a more “clinically relevant endpoint” and a sufficiently rigorous trial design will be more difficult, given the number of journals that publish clinical trial results and the number of government agencies that oversee drug and device approval in various countries.
That’s where the work of people like John Ioannidis will be most important. The first step in solving a problem is to identify it, and no one has been able to identify the problems with how science is applied to medicine as effectively and prolifically as John Ioannidis. I only wish he were more cognizant of the blind spot of EBM with respect to basic science than he appears to be.

Who Cares About Dr. Adam Cifu’s Plan to Fix EBM in Theory, When in Reality, He Elevated the MAHA Doctors Wrecking it Today?
No one forced Dr. Adam Cifu to volunteer as a defense attorney for MAHA doctors. He chose to do that all on his own.
Jonathan Howard on November 14, 2025
Link
Prasad’s lifelong commitment to scientific rigor will help rebuild lost trust in regulatory bodies.
Dr. Adam Cifu is a founder Sensible Medicine, a monetized misinformation Substack that published gems such as Why Doctors Should Learn to Stop Worrying and Love MAHA and The Nomination of Robert F. Kennedy Jr. as Secretary of the Department of Health and Human Services: A New Beginning? Sensible Medicine also celebrated the appointment of Dr. Jay Bhattacharya by saying:
The recent news that Dr. Jayanta Bhattacharya is being considered for the post of Director of the National Institutes of Health (NIH) was received with enthusiasm by many of us who opposed the US pandemic response. Jay, a Professor of Medicine, Economics, Heath Research and Policy at Stanford, was a primary author of the Great Barrington Declaration (GBD) and a consistent and articulate opponent of unscientific, harmful, and ineffective covid policies. He is a courageous, thoughtful, and deeply humane person who would make a great leader of any of our US public health agencies.
That wasn’t true, and predictably things aren’t going well at the NIH today.
However, Dr. Cifu similarly lavished MAHA doctors with praise, especially his friend Dr. Vinay Prasad. Sure, Dr. Prasad botched basic facts all the time- no COVID did not kill just 1 in a million children– but who cares whether or not doctors communicate accurate information about a deadly virus? Not Dr. Cifu, apparently.
Dr. Prasad openly campaigned for Trump and Kennedy, and Dr. Cifu barely batted an eye. He felt it was totally acceptable for doctors to elevate deluded anti-vaxxers to the highest offices in the land under the pretense this was the best way to save vaccines.
Dr. Prasad was cruel, juvenile, and profane. He sought to enrage online mobs, implying that beleaguered vaccine-hero Dr. Peter Hotez was indifferent to child sexual abuse, for example, and Dr. Cifu didn’t care at all. In fact, he thought it was funny that Dr. Prasad called doctors “motherfuckers” and “fucking morons” for not treating rare, temporary vaccine side effects as a fate worse than death from COVID.
While other doctors cited Dr. Prasad’s vulgar, vengeful, vitriol in articles on the fatal attack at the CDC, Dr. Cifu claimed this was effective, important science communication. In an essay defending this “style”, Dr. Cifu wrote:
This style, however, brought more attention to the issues he feels strongly about than the scores of sober papers he published. It probably also played a significant role in getting him appointed to a leadership position at the FDA, a role that seemed tailor-made for him.
Indeed, Dr. Cifu felt Dr. Prasad’s “style” and overt MAHA/MAGA activism meant there was “no better candidate” to help lead the FDA. Along with another misinformation superspreader, Dr. John Mandrola, Dr. Cifu endorsed him writing:
Prasad’s lifelong commitment to scientific rigor will help rebuild lost trust in regulatory bodies.
That wasn’t true, and predictably things aren’t going well at the FDA today.
He’s wreaking havoc with the staff. People are terrified.
And we should be clear about the stakes here. Not long ago, Drs. Cifu and Mandrola recorded an entire podcast with Dr. Prasad bemoaning a single canceled speech. With Dr. Prasad at the helm, many people’s entire careers have been cancelled. His brief tenure has been marked by high school drama, anti-vaccine fear mongering, purges, censorship, and outright lies about critics– namely myself. He avoids journalists, skips key meetings, and barely comes into the office. According to one article titled Under Vinay Prasad, Employees at A Key FDA Center Fear Speaking Out, Look for the Exits:
Dozens of scientists are considering leaving the Center for Biologics Evaluation and Research, where Prasad serves as director, to escape a work environment that eight agency officials described to STAT as rife with mistrust and paranoia. These officials said staffers are terrified of pushing back on Prasad, lest they face retaliation.
The fear has been fueled by Prasad pushing at least seven senior leaders out of their positions, employees told STAT, and offering no public explanations to staff or the leaders’ supervisors. Some of his workers are fed up, and are attempting to transfer to the FDA’s drug center. Prasad has tried to block these transfers in apparent recognition that CBER can’t afford to lose many more employees. The center has lost hundreds of employees since the start of this year to retirements, layoffs, and resignations, and hiring conditions are difficult.
CBER employees are concerned that the exodus of employees and the worsening morale problems will affect the functioning of the center, which ensures the safety and efficacy of vaccines and helps speed potentially lifesaving therapies to rare disease patients…
“He’s wreaking havoc with the staff,” one agency official said. “People are terrified.”
I’m glad I played no role in respected scientists being marched out of their office without explanation.
Another article, How Two Top FDA Officials are Quietly Upending Vaccine Regulations, reported that Dr. Prasad is censoring research showing results he doesn’t want to be true, namely that the COVID vaccine is beneficial.
In October, Prasad decided to withdraw the paper, which examined health outcomes among patients with private health insurance who received Covid vaccines from 2023 to 2024. The paper concluded that there were no new safety concerns following Covid-19 vaccination for people aged 6 months to 64, and that “the benefits of vaccination outweigh the risks,” according to a copy viewed by STAT.
“The retraction of the study marks a choice by leadership away from transparent and comprehensive reporting of the results of these government funded studies towards opaque and selective reporting of particular results,” a former employee of an FDA contractor told STAT.
The article also noted that, “the deputy director in the office of vaccines, quit the agency on Friday” and “The FDA, Prasad, and Høeg did not respond to STAT’s requests for comment, or provide answers to a detailed list of questions sent by email.”
Here are some other articles we are all reading about the FDA:
    • Experts Worry FDA’s Credibility Is Being Shredded by Scandal and ‘Soap Opera’
    • Dr. Prasad, the FDA’s Grim Reaper
    • FDA Described As “Clown Show” Amid Latest Scandal
    • The Spin Doctors at the FDA
    • Inside the Staff Exodus and Tanking Morale That Threaten Makary’s FDA
    • The FDA’s Meltdown Escalates
    • What to Know About the Collapse of the F.D.A
    • FDA Leaders Are “Acting Political”, Says Ex-Agency Chief Counsel
    • FDA Staff in Meltdown as RFK Jr. Chaos Triggers Mass Exit Fears
    • Drug Approvals Hit an FDA Wall FDA Review of Drugs Is Slowing While Application Delays Are Growing, Analysis Find
    • Fed-Up and Demoralized, FDA Staff are Leaving.
    • Can Clinical Trials Withstand the New FDA?
    • Inside FDA, Career Staffers Describe How Political Pressure Is Influencing Their Work
    • FDA Chaos Threatens Medical Innovation — and Patients’ Health
    • Chaos at the FDA Leaves Staffers Disturbed, Ready to Leave
Instead of rebuilding “lost trust,” confidence in our federal health agencies has tanked, and large swaths of the country are wisely ignoring their “guidance.” Talk about a medical reversal.
It’s nice to think there’s someone to blame, isn’t it?
While Dr. Cifu was happy to earn income from pro-Kennedy balderdash and fluff MAHA doctors himself, he purposefully undermined those of us who warned of their dangers. He repeatedly dunked on my writing, though he never even attempted a coherent refutation of even a single word. Instead of thoughtfully engaging with the arguments I made or the data I presented, he used haughty snark and sarcasm to breezily dismiss it all. It was bad faith enagement all the way down.
“It’s nice to think there’s someone to blame, isn’t it?” he said to me- and only to me- when I merely considered criticizing the authors of the Great Barrington Declaration, two of whom now hold prominent positions in the Trump administration. Apparently, influential advocates of herd immunity via mass infection, backed by a pro-tobacco, child-labor advocate and current MAHA powerbroker, deserved a safe space where they were shielded from even the slightest criticism.
Elsewhere, Dr. Cifu morphed into a music critic, fretting mightily about unspecified flaws in my “tone”. In a pandemic where over a million Americans died, many due to the misinformation I warned about, my decorum is what mattered most to him. “It is hard to engage when you feel like you’re being yelled at,” he said- again, only to me- without providing any examples. Imagine if I had created a collage of prominent doctors to shame and mock them on social media.
Other times, Dr. Cifu put his words in my mouth to make me sound like a blathering idiot and discredit my warnings about the threat of medical disinformation. He said to me:
I think it’s wonderfully optimistic to think had it not been for Drs. X, Y, Z everything would have gone swimmingly. You know our country as well as I do.
As people who’ve read my work know, I never said anything that stupid, though Dr. Cifu is not such a person. In fact, he made a great show of publicly refusing to even read an article of mine that corrected Dr. John Ioannidis’s baseless, nasty accusation that frontline doctors killed our patients through premature intubations. “It’s problematic when a paragon of evidence-based medicine claims vaccines spread disease and amplifies internet rumors,” I wrote. I stand by that. However, rather than even entertain the possibility that a peon like me could critique a hero of his, Dr. Cifu said that he had to run to clinic.
Trying to have a serious, mature conversation with him about evidence and science was impossible. I tried. I failed. I tried again. I failed again. It was like trying to teach a fish to ride a bicycle. It just wasn’t going to happen.
And while the disingenuous tone policing and deceptive straw man arguments were annoying, the core problem is that Dr. Prasad and I delivered very different messages. I warned that Kennedy might turn the CDC into an anti-vaxx propaganda outfit. In contrast, Dr. Prasad called the FDA a “corrupt entity that needs to be entirely renovated” and wrote articles such as Sabotaging RFK Jr’s Confirmation Will Increase Vaccine Hesitancy.
However, according to Dr. Cifu, the vulgar doctor who glorified Kennedy had a “lifelong commitment to scientific rigor” and deserved to lead CBER, while the doctor who sternly warned about Kennedy could be scolded and dismissed out of hand.
🔥 Dr. Vinay Prasad Praises RFK Jr

“On regulatory capture, on the COVID-19 vaccine, on the FDA being a corrupt entity that needs to be entirely renovated…I completely agree with him, and I wish him well in that pursuit.”@VPrasadMDMPH @RobertKennedyJr pic.twitter.com/MOTP7NQGH1
— Chief Nerd (@TheChiefNerd) October 30, 2024
Evidence-Based Medicine Requires Urgent Reform, Physician Says
Given this background, Dr. Cifu recently gave a lecture that caught my eye. It was summarized in an article titled Evidence-Based Medicine Requires Urgent Reform, Physician Says. He’s right. Would Dr. Cifu be bold enough to discuss what Kennedy and MAHA doctors are doing right now? Not according to the article.  It said:
Evidence-based medicine faces fundamental challenges that threaten its core mission of integrating clinical expertise with rigorous research, according to Adam Cifu, MD, who delivered a lecture at a Toronto physicians’ meeting. His analysis identified two competing explanations for what he calls evidence-based medicine’s failures: inherent design limitations predicted by early skeptics and systematic corruption by special interests.
Dr. Cifu identified several flaws in EBM: foundational flaws in trial design, post-randomization confounding, low positive predictive value, medical reversal phenomenon, and industry-influenced trial design.
The article concluded with Dr. Cifu’s proposed solutions:
Dr. Cifu outlined requirements for EBM restoration: “This demands unbiased research. It demands trials that are actually designed by actual physicians, practicing physicians, to answer important clinical questions that they identify.”
From NASA to the National Institutes of Health, federal agencies conduct research that universities cannot. Agency scientists speak out about the irreplaceable facilities, institutional knowledge and training opportunities that the country is losing.
Dr. Cifu’s talk, at least reported by the article, was mostly banal pablum, like saying kittens are cute and papercuts stink. Who doesn’t want “unbiased research?” and “trials that are actually designed by actual physicians”?
We all do, of course. The problem is that some of us can also read the news. We know what’s happening in science and medicine today and wonder who is going to carry out this “unbiased research” moving forward. Unless one feels Americans need to be protected from the grave threats posed by vaccines, food dyes in ice cream, wind farms, Tylenol, and chemtrails, we do not seem to be entering a golden age of biomedical research.
Dr. Cifu didn’t acknowledge any of this, however. There was no mention of any current events or the people responsible for them. His talk was a time capsule from a more innocent age, when Kennedy was a powerless crank, Dr. Ioannidis (whom Dr. Cifu positively referenced) was an unquestionable saint, and vital medical questions could be definitively answered if only we summoned the collective will to run the proper studies.
Dr. Cifu’s talk, it turns out, was just a dispatch from the pundit fantasy world. I previously used this term to describe sheltered pundits who acted as if all it took to open schools and protect the vulnerable was to utter the words “open schools” and “protect the vulnerable”. Maintaining this illusion required them to actively deny what happened in the real world when COVID was allowed to spread unchecked. Unwanted news was unmentionable news.
It’s super wonderful that Dr. Cifu wants “unbiased research,” but how does he feel things are going, not just at the FDA, but also the NIH and CDC? Can he make the affirmative case that these agencies are thriving under the doctors he defended and promoted?
 If not, his complaints about the current state EBM should be viewed as a searing indictment of their failed leadership. After all, they are the medical establishment now, and they are to blame if they don’t deliver the “unbiased research” Dr. Cifu claims to value. 
Perhaps I’ve missed it, but other than to deliberately distort Dr. Prasad’s pandemic record- yes Dr. Prasad repeatedly said “RCT or STFU” to undermine his predecessors- I’ve never seen Dr. Cifu seriously grapple with any of the headlines about the state of science in the U.S. today. Unwanted news is still unmentionable news.
Motivating biases need not be considered nefarious, only considered.
I have several proposals of my own to remedy this willful denial. Moving forward, anyone who opines on problems with EBM should, at a bare minimum, acknowledge what is happening to scientific research today and name the individuals responsible for it. It’s not that hard to simply be honest.
Furthermore, speakers should disclose if they opposed or enabled the people behind the headlines today, especially if they profited from articles like this. Our audiences deserve to know such things. That is as relevant as any pharma ties, and Dr. Cifu once wrote eloquently about the importance of considering such “motivating biases.” I agree.
My disclosures would read that I volunteered for a vaccine-RCT. Though that was a small contribution, it was a real one, and I am proud of it. Additionally, along with many others, I desperately tried to sound the alarm bells about Kennedy and MAHA doctors. We failed, but at least there’s a clear record of us doing everything we could to stop them.
I don’t think Dr. Cifu can’t say that.
As such, no one should care about his ideas to fix EBM in theory, when in reality, he volunteered as a defense attorney for the MAHA doctors taking a wrecking ball to it today. Though that was a small contribution, it was a real one. Nothing else about him should matter, and his empty pleas for “unbiased research” won’t change that.
And just to be clear, I totally agree with Dr. Cifu- tone is important. That’s why I’ve never said anything like this. I don’t think it is unfair for me to ask Adam to pause and reflect on that.


An antivaxxer attacks the evidence-based medicine paradigm: Echoes of RFK Jr.’s “gold standard science”
Last week, antivaxxer Toby Rogers published two very long screeds attacking the EBM paradigm. Cleverly, echoed some of our criticisms of EBM but twisted them so that they are science denying, rather than affirming—just like Robert F. Kennedy Jr.’s “gold standard science.”
David Gorski on July 21, 2025
Link
Longtime readers will know that Science-Based Medicine (SBM) was originally proposed as an answer to what we view as the shortcomings of the evidence-based medicine (EBM) paradigm. Indeed, let’s go way, way back to the very beginning of this blog—January 1, 2008!—when co-founder Dr. Steve Novella announced the new blog and briefly described out what SBM is. I think it’s worth quoting again, all these years later, as a prelude to discussing how a new mantra that has arisen, “gold standard science,” that is in reality an anti-science buzzword cleverly disguised as support for the best science that seems on the surface even to echo some of our criticisms of the EBM paradigm. My “inspiration,” if you will, for this post were two very long screeds—so long, in fact, that they make my tendency towards logorrhea seem like the most economical use of language you can imagine!—by an antivaxxer named Toby Rogers. (Seriously, the first Substack I will address is close to 9,800 words, and the second is over 5,000 words, both not counting references.) I’ll have more to say about him later, but for now suffice to say that I’ve written about Toby Rogers a number of times although mainly on my not-so-super-secret other blog, and, unsurprisingly, he is now a fellow at the Brownstone Institute. The two Substacks by Rogers that I will be addressing include How Big Pharma hijacked Evidence-Based Medicine, Part I (subtitled Evidence-Based Medicine is not evidence-based nor medicine) and part 2, Seven philosophical criticisms of Evidence-Based Medicine and evidence hierarchies.
First, by way of comparison, let’s look at what Dr. Novella wrote about SBM and EBM in the very first post published on this blog:
Within the practice of medicine there is already a recognition of the need to raise the standards of evidence and the availability of the best evidence to the practitioner and the consumer – formalized in the movement known as evidence-based medicine (EBM). EBM is a vital and positive influence on the practice of medicine, but it has its limitations. Most relevant to this blog is the focus on clinical trial results to the exclusion of scientific plausibility. The focus on trial results (which, in the EBM lexicon, is what is meant by “evidence”) has its utility, but fails to properly deal with medical modalities that lie outside the scientific paradigm, or for which the scientific plausibility ranges from very little to nonexistent.
All of science describes the same reality, and therefore it must (if it is functioning properly) all be mutually compatible. Collectively, science builds one cumulative model of the natural world. This means we can make rational judgments about what is likely to be true based upon what is already well established. This does not necessarily equate to rejecting new ideas out-of-hand, but rather to adjusting the threshold of evidence required to establish a new claim based upon the prior scientific plausibility of the new claim. Failure to do so leads to conclusions and recommendations that are not reliable, and therefore medical practices that are not reliably safe and effective.
This is why the authors of this blog strongly advocate for science-based medicine – the use of the best scientific evidence available, in the light of our cumulative scientific knowledge from all relevant disciplines, in evaluating health claims, practices, and products.
A number of our early posts started to put some meat on the bones of this concept, describing, for instance, how EBM as a paradigm for evaluating scientific evidence almost completely ignored—and, to a disturbing extent, still does—the concept of prior plausibility, thus allowing alternative medicine modalities whose precepts are, for all intents and purposes, physically impossible (e.g., homeopathy) to appear to have activity in the apex of single-study EBM levels of evidence (double-blind randomized controlled clinical trials, commonly abbreviated RCTs). Thus were deceptively justified abominations such as “integrative medicine,” quackademic medicine, and the National Center for Complementary and Integrative Health (NCCIH), formerly the National Center for Complementary and Alternative Medicine (NCCAM) over the last three decades. At the time, we advocated for a more Bayesian–based approach to RCT evidence that could take into account prior plausibility. Later, I started to discuss the ways that a hyper-strict reading of the EBM paradigm—which I like to refer to as “methodolatry,” or the “profane worship of the RCT as the only valid method of investigation”—had been increasingly weaponized by COVID-19 contrarians, pandemic minimizers, and antivaxxers to cast doubt upon vaccines and public health interventions, an approach that the antivax activist, Robert F. Kennedy Jr., who is now Secretary of Health and Human Services, has continued under the guise of the administration’s buzzword “gold standard science.”
Reading Rogers’ screeds, one thing became apparent to me immediately. Whereas we at SBM have criticized EBM in order to strengthen the science behind medical tests and interventions and eliminate implausible treatments that do not work but can produce false positives in RCTs (e.g., homeopathy and acupuncture), Rogers’ purpose is very different. His criticisms of the shortcomings of EBM are not made as a prelude to trying to preserve the best of what EBM offers and to propose modifications to the EBM paradigm—as we have with the SBM paradigm—to strengthen the scientific evidence base undergirding our medical interventions. Rather, he is making these attacks to call into question what EBM tells us as a prelude to implying that his antivax ravings and quackery must therefore be taken seriously and that EBM is nothing more that a big pharma psyop to protect its hegemony and profits. It’s an old criticism of EBM, that it’s been “hijacked” by big pharma; indeed, John Ioannidis made just that claim nine years ago. Let’s just say that Rogers is no Ioannidis. (Sadly, these days Ioannidis is no Ioannidis any longer.)
Toby Rogers attacks EBM, part I
A superficial reading of Rogers’ first ten criticisms of EBM might lead the reader to think that there is substantial agreement between the SBM crew and Rogers. For instance, Rogers criticizes the evidence hierarchy used in EBM; so have we, for instance, the way that basic science evidence is placed near the very bottom of the EBM pyramid of evidence, when scientific plausibility needs to be a major consideration when considering various alternative medicine modalities. However, it doesn’t take long to see what Rogers is about. While our critique of EBM is basically that EBM is a good basic framework that works for a large percentage of cases (e.g., testing whether pharmaceutical drugs work or not), it is not enough and does not go far enough. Indeed, our key criticisms, again, boil down to:
     1.  EBM’s lack of consideration of scientific prior plausibility, which can allow pure quackery to be portrayed by advocates of “integrative medicine” as “evidence-based”; e.g., homeopathy, acupuncture, and the like. Then advocates use RCTs that seem to show these modalities to be effective as a means of “integrating” quackery with real EBM, or, as Dr. Mark Crislip so famously put it, to “integrate” cow pie with apple pie.
     2.  How some EBM fundamentalists (e.g., Dr. Vinay Prasad) have weaponized the status of RCTs at the top of the EBM pyramid to cast doubt on public health interventions that are not easily amenable to being studied by RCTs or for which RCTs would be unethical. The most famous example is RFK Jr.’s call for “double-blind” placebo controlled clinical trials of every vaccine on the childhood schedule, deceptively casting the lack thereof for some vaccines as evidence that they are dangerous—or, at the very least, inadequately studied—rather than as a result of its being unethical to randomize clinical trial subjects to a saline placebo control group when an effective vaccine exists.
Let’s compare the above to Rogers’ ten criticisms, which range from the seemingly reasonable to the more ideological:
     1.  EBM has become hegemonic in ways that crowd out other valid forms of knowledge;
     2.  Evidence hierarchies do not just sort data, they legitimate some forms of data and invalidate other forms of data;
     3.  Meta-analyses and systematic reviews of RCTs are beset with epistemic problems;
     4.  Most RCTs are designed to identify benefits but they are not the proper tool for identifying harms;
     5.  RCTs are designed to address selection bias but other forms of bias remain;
     6.  Case reports and observational studies are often just as accurate as RCTs;
     7.  EBM is not based on evidence that it improves health outcomes;
     8.  EBM and evidence hierarchies reflect authoritarian tendencies in medicine;
     9.  Evidence hierarchies have reshaped the practice of medicine for the worse; and
     10.  Evidence hierarchies objectify and/or overlook patients.
Ironically, #1 and #2 rather echo one of my criticisms, #2 more than #1, but Rogers is not about what we at SBM are about. Indeed, #8-#10—give the game away, in particular the lament that EBM and evidence hierarchies are “authoritarian” in nature and that EBM “objectifies” and/or “overlooks” patients, thus “reshaping the practice of medicine for the worse.” Shades of the “health freedom” movement! To put it explicitly, Rogers is not interested in making rational changes in order to build up the edifice of EBM to make the evidence base behind medicine stronger, he’s about tearing the edifice of EBM down, so that its walls have gaping holes in it, the better to let quackery and his antivax pseudoscience in. Indeed, one thing that I noticed immediately is that few of the articles that Rogers cites are less than a decade old, and many are more than two decades old. Thus, his arguments seem to be against an older, early version of EBM, as if EBM has never been updated. For example, he takes umbrage at arguments made in a 2004 commentary by Brendan M. Reilly, in which Reilly wrote:
Few would disown the EBM hypothesis — providing evidence-based clinical interventions will result in better outcomes for patients, on average, than providing non-evidence-based interventions. This remains hypothetical only because, as a general proposition, it cannot be proved empirically. But anyone in medicine today who does not believe it is in the wrong business
However, Reilly also wrote in the same commentary:
But it is the third essence of EBM—the process of practising it—that we understand least and spar about most. Many of us teach EBM (integrating best evidence with clinical expertise and patient values) knowing that it is nearly impossible to practise it in everyday clinical care.8–10This makes sense because someday practising EBM will be feasible—when technology for searching literature improves (soon) and when all of us are more EBM facile (not so soon). But we should not promote the practice of EBM until we know whether the process itself improves patients’ care. Improves? Compared with what? In our laudable compulsion to translate more and better research faster into clinical practice, we should keep in mind that mortality rates for many diseases have dropped dramatically in the past few decades—for example, a 50% decline in cardiovascular deaths in the United States.11 So someone must be doing something right. But who and how? The great irony about promoting the practice of EBM in the future is that we know so little about how clinicians practise medicine in the present.10,12 We need to find this out, if only to establish credible comparison groups for the experiments that must be done. When this research is done, one thing seems certain: at least some of us will feel good about the road we have taken.
Which is a lot more “gray” and less dogmatic than how Rogers portrays it as too “strident.” He prefers instead an argument made by D. M. Berwick in a 2005 commentary:
…we have overshot the mark. We have transformed the commitment to “evidence-based medicine” of a particular sort into an intellectual hegemony that can cost us dearly if we do not take stock and modify it. And because peer reviewed publication is the sine qua non of scientific discovery, it is arguably true that hegemony is exercised by the filter imposed by the publication process…
My first reaction was: Sir, you say this as though peer-reviewed publication being the main basis for what is accepted as scientifically supported in medicine were a bad thing. The ellipse at the end also led me to look up the commentary to see what Berwick wrote next, which was to refer to refer to an article in the same issue by Davidoff and Batalden. What was that article about? Basically, it was a lament that research into quality improvement (QI) was seldom published compared to science, along with reasons why and suggestions for improvement. For example, Davidoff and Batalden noted that many QI investigations could often fall prey to The Common Rule and be considered human subjects research and thus subject to institutional review board oversight and other onerous requirements that are arguably not necessary for research into improving processes of medical care rather than testing interventions. It’s a subject that I know a bit about, having spent several years involved in a statewide QI project designed to identify problems with breast cancer care and identify strategies to improve it. In other words, Davidoff and Batalden weren’t rejecting EBM; rather, they were proposing publication guidelines for QI research. Moreover, all QI research starts with evidence-based guidelines of care.
I could go on, but you get the point. Moving on to the second criticism, I could well have some sympathy for this point of view if I didn’t know what Rogers’ game was really about. I mean, seriously, I almost could have written this passage by Rogers:
Although EBM in the early years made reference to the totality of evidence, soon EBM became a way of excluding all studies except double-blind, randomized, controlled trials (RCTs) from the analysis. Stegenga (2014) writes: “The way that evidence hierarchies are usually applied is by simply ignoring evidence that is thought to be lower on the hierarchies and considering only evidence from RCTs (or meta-analyses of RCTs).”
Sound familiar? Yes, I myself (and others here at SBM) have criticized what I sometimes call “EBM fundamentalists” for their “methodolatry,” a term that I first discovered way back in 2009, and, once again, defined as the “profane worship of the randomized clinical trial as the only valid method of investigation.” Indeed, I’ve been noting the tendency towards methodolatry in EBM mavens dating back at least that far, ranging from prominent members of the Cochrane Collaborative, such as Tom Jefferson and Peter Gøtzsche, to famous scientists like John Ioannidis, to, of course, COVID-19 contrarians like Vinay “RCT or STFU” Prasad. In his other Substack article, Rogers even echoes our earlier criticism of EBM’s blind spot towards basic science evidence. (More on that later.) I had to ask: Am I dreaming? Have I gone mad? An antivax and transphobic advocate for quackery just wrote something that I might have written! Interestingly, though, he doesn’t really expound upon the theme, other than to quote a couple articles from 2005 and 2014. Moreover, if Rogers were really about criticizing methodolatry and EBM fundamentalism “excluding” all studies other than double-blind placebo-controlled RCTs, you might think he’d have a problem with RFK Jr.’s call for studying every vaccine on the current schedule in an RCT versus saline placebo. If he has made such a criticism, I haven’t found it, but I have found his praise for RFK Jr., enough to have been a regular contributor for a time to RFK Jr.’s antivax org Children’s Health Defense.
Of course, Rogers also attacks meta-analyses. Once again, as we have said here at SBM more times than I can remember, when it comes to meta-analyses, “garbage in = garbage out” (GIGO). No one denies that meta-analyses can be biased, but Rogers’ discussion relies on, once again, old publications, many of which date back to before PRISMA guidelines were first published. PRISMA guidelines, recall, were formulated to minimize or eliminate many of the biases that Rogers discusses. Since then, PRISMA guidelines have been continually updated, with the most recent version being PRISMA 2020 and most journals requiring systematic reviews and meta-analyses to follow PRISMA guidelines. Rogers might have a point when he states:
It is not that subjectivity itself is necessarily a problem. The subjective wisdom that comes from years of experience could be quite helpful in evaluating the evidence. The problem with meta-analyses as currently practiced is that those involved usually do not acknowledge their own subjectivity while simultaneously excluding the sort of reasoned subjective analysis (from doctors, patients, or perhaps even philosophers) that might be helpful.
However, once again, notice how he calls for more “subjective analysis,” rather than efforts like PRISMA to make meta-analyses more objective. One can’t help but acknowledge that no effort of human beings can ever be totally objective or free from subjective influences, but what Rogers is calling for is not greater rigor, but more of a window to allow subjectivity in. Basically, he wants to replace what he sees as the influence of big pharma with the influence of people like him, and I know his history with respect to science and vaccines.
The same is true regarding #5. Yes, RCTs are best at minimizing selection biases, and, yes, RCTs can have other biases. Perhaps the most telling part is this:
So simply controlling for selection bias is not sufficient to guarantee scientific integrity. Furthermore, it is not even clear that RCTs as currently practiced actually prevent selection bias:
A research group conducted a systematic review of 107 RCTs about a particular medical intervention, using three popular QATs [Quality Assessment Tools] (Hartling et al. 2011). This group found that allocation concealment was unclear in 85% of these RCTs, and that the vast majority of the RCTs were at high risk of bias. Another group randomly selected eleven meta-analyses involving 127 RCTs on medical interventions in various health domains (Moher et al. 1998). This group assessed the quality of the 127 RCTs using QATs, and found the overall quality to be low: only 15% reported the method of randomization, and even fewer showed that subject allocation was concealed (Stegenga, 2015).
Perhaps the authors of these studies were simply careless in describing their methods. But given that directors of Contract Research Organizations boast of their ability to deliver the results desired by their clients (Petryna, 2007 in Mirowski, 2011) it seems reasonable to wonder whether double-blind randomization is actually happening at all in some clinical trials that purport to be RCTs.
I would counter with a different take on this problem and argue this: The problem, as related by Rogers, is not with RCTs per se, although he wants to make you think that it is. Rather, the problem is that all too often we as a community of physicians and scientists do not carry out and report RCTs in a sufficiently rigorous fashion and that perhaps greater oversight is required over CROs. After all, the reason that pharmaceutical companies hire CROs is because CROs can allow them to open a clinical trial in many locations and allow them to contract out the work of actually recruiting subjects and running the actual trials that they’ve designed in consultation with the FDA. Again, it is telling that Rogers doesn’t suggest any strategies to correct this perceived problem. He only enumerates the problem in order to cast doubt on the findings of EBM based on RCTs.
The same thing is true of #6, which notes that case reports “and observational studies are often just as accurate as RCTs.” Of course, in this case, the word “often” is doing a lot of heavy lifting here, but never let it be said that Toby Rogers was ever in any way constrained by complexity or an obligation to look at a question more holistically. For instance, to ask whether case reports are as accurate as RCTs, Rogers dips way, way back to a study from 1982, where the investigators looked at 52 first reports of adverse reactions to drugs, following up 18 years later to see if the adverse reactions were validated as having been due to the drug suspected in larger studies, noting that 75% of the adverse reactions were later confirmed in additional studies as having been due to the drug. This then leads Rogers to opine:
When one compares the 75% success rate of anecdotal first reports with the fact that 75-80% of the most widely cited cancer RCTs cannot be replicated (Prinz, Schlange, and Asadullah, 2011; Begley and Ellis, 2012), the decision to place RCTs at the top of the CEBM evidence hierarchy, while denigrating case series, appears unwarranted.
This is totally a bait-and-switch, because the articles cited do not answer the question of the reliability of RCTs. For example, Prinz et al is a study looking at preclinical studies identifying molecular targets for drugs and has nothing to do with clinical trials. It looked at preclinical drug discovery in cell culture and animal models; it’s about “wet lab” science, not clinical science, as the authors themselves write. Similarly, the title of the Begley and Ellis article tells you what it’s about, namely, Raise standards for preclinical cancer research. It’s basically the same message, that a lot of preclinical science, warning glumly that the “inability of industry and clinical trials to validate results from the majority of publications on potential therapeutic targets suggests a general, systemic problem.” In other words, both papers are about the difficulty and lack of reproducibility translating preclinical science into actual cancer therapies; these articles have nothing to say about the reliability of RCTs compared to observational studies. Nice try, Toby. You thought no one would actually look up your references and read them.
But coming back to the question of whether observational studies versus RCTs, Rogers actually cites a paper that points out that it’s far to simplistic a question to ask if, in general, whether observational studies can be as accurate as RCTs. As noted by Bosdriesz et al in 2020, who in a review of this very question concluded, quite reasonably:
The answer to the question “is an observational study better than an RCT?” depends on the research question at hand. There is not one overall gold standard study design for clinical research. With respect to analytic studies, the RCT is the best study design when it comes to evaluating the intended effect of an intervention. However, this type of research represents only a fraction of all clinical research. Observational analytic studies are most suitable if randomization of the intervention or exposure is not feasible or if the research question focuses on unintended effects of interventions. For non‐analytic or descriptive studies, observational study designs are also needed. We conclude that RCT and observational studies are inherently different, and each have their own strengths and weaknesses depending on the study question.
Rogers take on the article? This:
In 2017, Thomas Frieden, the former Director of the CDC, made the case in the New England Journal of Medicine that a wide range of different study types can have a positive impact on patients and policy. He makes the simple point that each type of study has strengths and weaknesses and the study type should match the type of problem the researchers are trying to address. He points out that alternative data sources are “sometimes superior” to RCTs.
So a wide range of different types of evidence can be valid and help inform clinical decision-making and yet the current practice of EBM systematically excludes everything other than the large RCTs favored by pharmaceutical companies.
No one other than EBM fundamentalists—and perhaps not even EBM fundamentalists—argues that we shouldn’t evaluate a range of different types of evidence. Remember that the EBM paradigm is about ranking the relative strength and reliability of different forms of evidence, not excluding them; at least, that’s how it’s supposed to work. Again, given what Rogers is arguing, one might think that he’s unhappy with the “gold standard science” being touted at HHS that supposedly mirrors Dr. Prasad’s longstanding (at least since the pandemic and only about public health) “RCT or STFU” ethos.
I also can’t help but be amused a bit at #4, namely that “RCTs are designed to identify benefits but they are not the proper tool for identifying harms.” That’s not entirely true. RCTs are a perfectly acceptable tool to identify harms. Every RCT that I read reports the rate of adverse events in the control and experimental arms and assesses whether there are specific adverse events that are more common in the treatment group. The problem with RCTs with respect to identifying harms is not that they can’t identify harms. It’s that they are not good tools for identifying rare harms, mainly because there are usually not enough subjects in each group to detect statistically significant differences in such uncommon harms. That’s why pharmacovigilance programs exist.
I will concede that Rogers has a point when he notes that “FDA post-market surveillance is under-funded by design and not sufficiently staffed to respond to the size of the task,” except that I don’t think it’s by design or intent. It’s more as a result of neglect and politics. It’s because the FDA has long been underfunded for all its activities, a situation that is about to become very much worse. If Rogers is really concerned about post-licensure surveillance of, for instance, vaccines, he should be outraged at the massive budget cuts aimed at the FDA for fiscal year 2026. If he has expressed such outrage, I have yet to see it. Maybe a reader could clue me in.
Oh, wait. Maybe not:
Riddle me this: for the last 30 years, Evidence-Based Medicine (EBM) has extolled the virtues of randomized controlled trials (RCTs). Some of the biggest blockbuster drugs over this time period include vaccines, SSRIs, and statins. But the so-called “RCTs” for those drugs were all completely fraudulent and EBM said nothing. So what exactly is EBM if they celebrate clearly rigged data as the “highest level of evidence”?
– Toby Rogers
Read on Substack
From my latest article:
– Toby Rogers
Read on Substack
How do we know they’re fraudulent? Rogers doesn’t say, other than to say:
Everyone knows that they are fraudulent (even though the mainstream medical profession tries to excuse this fraud). In clinical trials for vaccines, the control group is not given an inert saline placebo and is given another toxic vaccine or the toxic adjuvants from the trial vaccine instead. The Informed Consent Action Network (2023) has the receipts.
But wait. I thought that Rogers’ complaint was that EBM was too hidebound and too dogmatic in demanding RCTs über alles. I’m so confused. Of course, Rogers’ real game is painfully obvious. He doesn’t like EBM because he thinks all the RCTs on vaccines that demonstrate their safety and efficacy are “fraudulent. Ditto the RCTs for drugs that he doesn’t like, such as psychiatric drugs and statins. Let’s just say that, as a scientist, physician, and/or clinical trialist, Rogers is a political economist.
The remaining complaints against EBM are pretty standard quack complaints about conventional medicine in general, particularly the claim that EBM is authoritarianism. At this point, I was getting similar vibes that I got nearly two decades ago when I first encountered quacks arguing against EBM by calling those who demand EBM, specifically the Cochrane Collaboration, “microfascists,” to which I added that they must also be wearing micro-brown shirts and black boots and carrying micro-truncheons. Rogers also complains that pharmaceutical companies use EBM to sell their products, to which I responded: Damn those evil pharma companies, selling their products by touting that they had been shown to be safe and effective in RCTs! Those bastards!
Naturally, Rogers can’t resist a bit of a conspiracy theory. He notes how N-of-1 trials (trials of a single patient observed before and after an intervention) were in 2002 proposed to be at the top of the evidence hierarchy by the American Medical Association (AMA). However, over the next 13 years, N-of-1 trials kept falling down the ladder of the evidence hierarchy, so that by 2015, the AMA had fully embraced Grading of Recommendations Assessment, Development, and Evaluation (GRADE) as the system for ranking strength of evidence:
The third edition of The Users’ Guide to the Medical Literature published in 2015 fully embraces GRADE as the AMA’s preferred framework for making prevention and treatment decisions.
I saw GRADE in use when I watched every meeting the FDA’s Vaccines and Related Biological Products Advisory Committee (VRBPAC) and the CDC’s Advisory Committee on Immunization Practices (ACIP) in 2022 and 2023. GRADE is a tool to give legitimacy to ANY medical intervention no matter how abysmal the data.
And:
So within 13 years (from the first edition in 2002 to the third edition in 2015) the AMA went from the best-in-class evidence hierarchy that acknowledged individual difference to a cartoonish monstrosity, GRADE, that is just a tool for laundering bad data on behalf of the pharmaceutical industry. In the process the AMA sold out the doctors in their association and the patients in their care to the drug makers.
There you have it, the real reason why Rogers detests GRADE and the trappings of EBM; they were the basis for approving COVID-19 vaccines, and they are at the basis of all the clinical data showing that vaccines are safe and effective.
Rogers goes Bayesian and basic science?
As I said above, it both alarms and amuses me that Rogers could write things that sound like something I might have written, even if it’s only short passages of a paragraph or two. For example, in his second Substack on the topic, he writes:
This inferential gap is unlikely to ever be bridged because there is infinite variety in the human population so responses to medical interventions will always vary as well. Bayesian statistics might help narrow the gap a bit (see next section) as it allows one to continually refine estimates as new evidence becomes available (conditional probabilities that affect the prior probability of the hypothesis). But even with Bayesian statistics, the best one can come up with are probabilities, not the deterministic thinking of EBM. These are extraordinary debates at the core of the philosophy of medicine — and it is exactly these sorts of debates that EBM proponents circumvent in making RCTs the sole tool for clinical decisions.
As I discussed earlier in this post, a key component of SBM, which is to us an enhancement of EBM, is the use of Bayesian inference, which requires an estimate of prior probability. In other words, when doing an RCT, one needs an estimate based on existing evidence of what the probability is that there will be a significant difference between the control group and the treatment group. Our point was not so much to argue over fine differences in prior probability but rather to point out that, for modalities that are physically impossible, like homeopathy, the prior probability is very close to zero, if not for all intents and purposes indistinguishable from zero. The consequence of such reasoning is that there are some treatments that can be rejected on the basis of basic science alone and that for treatments with an incredibly low prior probability truly extraordinary evidence of efficacy would be needed to accept them.
Rogers misleading portrays the battle between frequentist statistics and Bayesian reasoning as the sole reason why RCTs rule. Yet there is no reason why Bayesian statistics can’t be used to design clinical trials and analyze their results. Indeed, this has been proposed by simply coming up with a table over what p-value should indicate statistical significance in an RCT based on the estimated prior probability, with lower prior probability hypotheses requiring a much lower p-value to be considered significant, a Bayesian correction, if you will. The real tricky part is coming up with a reasonable estimate of prior probabilities.
Now here was the disturbing passage, if only because it sounds as though it could have been written by yours truly:
Several authors have noted that EBM tends to overlook and ignore the contributions of basic science (also called “bench” or “fundamental” science and I will use these three terms synonymously in this section). Bench research is defined as “any research done in a controlled laboratory setting using nonhuman subjects. The focus is on understanding cellular and molecular mechanisms that underlie a disease or disease process” (“Bench research”, n.d.). Merriam Webster’s Dictionary defines basic science as, “any one of the sciences (such as anatomy, physiology, bacteriology, pathology, or biochemistry) fundamental to the study of medicine” (“basic science”, n.d.).
The CEBM evidence hierarchy lists basic science as the fifth level of evidence, below the threshold suggested by Strauss et al. (2005) and others as even worth reading. To be clear, the CEBM and other evidence hierarchies are not excluding bench science entirely from the study of medicine — they are proscribing the consideration of bench science by doctors when they make clinical decisions (presumably others, namely pharmaceutical companies and academic researchers would be free to continue with a more comprehensive approach). Excluding basic science in this way is an odd choice because basic science has always been an essential component of establishing causation.
Even as much as this sounds disturbingly like what we’ve argued here on SBM, there are enough “tells” to let you know that Rogers is going in a much different directions. (It also wouldn’t surprise me if the “several authors” referred to above include Steve Novella, Kimball Atwood, and myself, all of whom have written on this topic, including in the peer-reviewed biomedical literature. In any case, Rogers continues:
Goldenberg (2009) argues that the degradation of pathophysiology in evidence hierarchies is unwarranted “as pathophysiology often provides more fundamental understanding of causation and is in no way scientifically inferior” (p. 180).
La Caze (2011) voices alarm that evidence hierarchies overlook the contributions of basic science. As pointed out above, basic science is usually assigned to the lower tiers of evidence hierarchies. While the assignments to the different tiers are rationalized based on reference to “quality” in fact, “proponents of EBM provide little justification for placing basic science so low in EBM’s hierarchy” (La Caze, 2011, p. 96). “Proponents of EBM urge clinicians to base decisions on the outcomes of large randomised studies rather than the mechanistic understanding of pharmacology and physiology provided by basic science” (La Caze, 2011, p. 83).
Again, when we at SBM invoke basic science, we tend to invoke it in a negative way, where certain modalities that are physically impossible or so close to impossible as to be indistinguishable from it (e.g., homeopathy) can be rejected solely on their violating the laws of physics and where modalities that are incredibly improbable (e.g., acupuncture) require extraordinary evidence to overcome their near-zero prior probabilities. I suspect that what Rogers is signaling is approval of “autism biomed” quackery, which often treats autism based on dubious biochemical mechanisms that its practitioners apply to the condition of autism whether they are relevant to the disorder or not. He might also be making a nod to “functional medicine,” which worships “biochemical individuality” as an excuse to justify a clinician doing whatever they want, regardless if it’s science- or evidence-based or not.
What this is really about
What Rogers is about has little or nothing to do with strengthening the scientific basis of medicine or improving medicine care. Rather, it’s about giving quacks and antivaxxers cover under the guise of science to do whatever they were going to do anyway. To show this, it’s helpful to quote the last passage in each of Rogers’ Substacks.
From part I:
We know that EBM is a fraud because it ranks rigged corporate studies ahead of the paradigm-shifting breakthroughs discovered by parents that are actually helping autistic children.
Remember Jenny McCarthy referring to her son whose autism she blamed on vaccines, saying “Evan is my science”? That’s what Rogers wants. Also:
Going forward, any system of medicine in connection with autism must start with the individual child and his/her family as the highest form of evidence (because obviously they are). All forms of data, no matter how unconventional or “outside the box,” must be brought to bear on supporting recovery and preventing this injury from happening to others. Rigged corporate RCTs have no place in actual medicine; their only appropriate use is as evidence of crimes against humanity in future Nuremberg trials of pharmaceutical executives and their enablers in government. The revolution we seek is thus a return to actual science instead of the genocidal corporate nonsense posing as evidence-based medicine today.
Wow. Rogers is an angry dude who hates EBM. Don’t believe me? Here’s the conclusion of his second Substack, or part 2:
The history of EBM reads like a Greek tragedy. A bunch of smart, seemingly well-intentioned people organized themselves to take over the practice of medicine. They wanted to make it better. It went well for a while but then hubris, greed, power, and corruption took over. Epidemiologists became a new priestly class and replaced science with dogmatism. Once unleashed, EBM became a runaway freight train. Now it is actively harming patients and destroying allopathic medicine in the name of saving it.
We need not sacrifice our dignity, common sense, and rational faculties on the altar of EBM as Guyatt and others have done. Rigged RCTs are not evidence. Corporate science is not science. We need to return to the old ways. We must let doctors be doctors again, relying on evidence, experience, and intuition — phronesis as Aristotle taught us (and Kathryn Montgomery reminds us). And we must let parents be parents again. Personal sovereignty and responsibility are the foundation of medicine and society. No financially conflicted epidemiologist in an ivory tower thousands of miles away (or heaven help us, Washington D.C.) knows what’s best for a person. The era of corporate EBM is over and the future of medicine is decentralized, N-of-1, non-corporate, non-government, person-to-person, direct primary care, based on the totality of evidence, decency, life experience, dialogue, and personal values.
Does the term “actual science” remind you of RFK Jr.’s and cronies’ parroting the term “gold standard science.” It should. It’s the same concept in a different form. Of course, while RFK Jr. and his acolytes like Vinay Prasad and Marty Makary like to imply that this “gold standard science” involves the most rigorous application of the EBM paradigm, up to and including saline placebo-controlled RCTs of every vaccine on the childhood vaccination schedule, when it comes to interventions and quackery that RFK Jr. tends to like, that “gold standard science” eagerly embraces a “more fluid” concept of evidence.
That is what Rogers is really embracing. Not “actual science.” Not a the actual meaning that real scientists would take from the term “gold standard science.” Rather, he wants to base medicine on any form of science and evidence that confirms what he already believes, namely that vaccines cause autism, are ineffective, and are downright dangerous. He wants to base medicine on whatever bias-prone personal observations that parents or quacks have regarding autistic children or any other person with a medical condition. Any other form of evidence is to him “rigged” or “corporate.”
Far be it from me to deny that there are problems with the EBM paradigm. After all, I’ve written about the problems that I perceive with EBM for over 17 years now. The difference between my criticisms and Rogers’ criticisms is that mine are grounded in actual science (if you’ll excuse my appropriation of the term), while Rogers are designed to let quacks find whatever “evidence” they need to ply the quackery that they were going to ply anyway, just now with seeming “science” behind it.
    • 

Clinical trials of homeopathy versus “respect for science”
David Gorski on March 9, 2015
Link
A few months ago, Steve Novella and I published an article in Trends in Molecular Medicine entitled “Clinical trials of integrative medicine: testing whether magic works?” It was our first foray together into publishing commentary about science-based medicine versus evidence-based medicine, using a topic that we’ve both written extensively about over the years on this blog and our respective personal blogs. Specifically, we discussed whether it is worthwhile to do randomized clinical trials (RCTs) testing highly improbable treatments, such as reiki and homeopathy, both of which have no physical basis to believe that they do anything whatsoever. As I’ve said many times before, reiki is simply faith healing in which Eastern mysticism is substituted for Christian beliefs, and homeopathy, as we’ve discussed many times here on SBM, is vitalistic sympathetic magic with no evidence to support its two laws.
To our surprise, that article generated a fair amount of press (for example this), with accounts of it showing up in the media in various places and Steven and I being asked to do a fair number of interviews. Part of the reason, I suspect, is that the editor made the article available for free for a month after its initial publication. (Unfortunately it’s back behind the pay wall again.) Part of the reason is that, intuitively, it makes sense to people not to waste money testing what is, at its core, magic. When I followed up that publication with an article criticizing “integrative oncology” in Nature Reviews Cancer entitled “Integrative oncology: Really the best of both worlds?“, the target was well and truly on my back. Indeed, let’s just say that the Society for Integrative Oncology and the Consortium of Academic Health Centers for Integrative Medicine (CAHCIM) are quite unhappy with me. When both their letters to the editor are published (right now, only one is), I might even blog about them.
In the meantime, I want to deal with criticism published in an unexpected place, albeit not by unexpected critics. The reason is that this criticism relies on a common straw man caricature of what we are saying when we advocate science-based medicine (SBM) that considers prior plausibility in determining what modalities to test in clinical trials and understands Bayesian thinking in which prior plausibility affects posterior plausibility that a “significant” result is not a false positive in contrast to the current evidence-based medicine (EBM) paradigm, which relegates basic science knowledge, even well-established principles of science that show that something like, say, homeopathy or reiki is impossible under the current understanding of physics, chemistry and biology, to the lowest rung on the EBM pyramid. It’s also a criticism that comes up frequently enough that, even though it’s been addressed before in various ways by various SBM bloggers, it’s worth revisiting from time to time. In this case, that’s particularly so because one of the two critics taking Steve and me to task is currently embroiled in a controversy about testing homeopathy for attention deficit hyperactivity disorder (ADHD) at the University of Toronto (more details on that later). Let’s just say, the criticism of Steve and me gives me an “in” to address a story that I thought had passed me by, and I intend to take it.
Steve and I are criticized as unscientific
I mentioned that I was surprised about where this criticism of Steve and me was published; the reason for my surprise is that it was published in Focus on Alternative and Complementary Therapies (FACT), a journal edited by Edzard Ernst. This is usually a journal that is—shall we say?—not particularly hospitable to arguments like this. Not this time. Witness the article by Sunita Vohra and Heather Boon entitled “Advancing knowledge requires both clinical and basic research“. The criticism begins thusly:
Recently, Gorki [sic] and Novella [1] recommended an approach that would severely curtail one’s ability to explore novel therapies and gain new understanding about human biology. They propose that ‘science-based medicine’ replace the modern standard of ‘evidence-based medicine’ and be used to limit clinical investigation to only those therapies that have positive basic science studies.[1] This approach is a cause for concern as it is predicated on an assumption that we already understand how the world and, in particular, the human body, works. This seems to fly in the face of the basic scientific method, which starts with an intriguing observation and encourages one to ask ‘why did that happen?’ The point of science is to explore things that challenge our understanding and current paradigms of how things work. Theories need to be revised in light of available data, rather than be used to curtail the kinds of questions one can ask.
I had wondered why this article had not popped up on my Google Alerts feed before given that, in order to monitor what people are saying about me and where I’m being mentioned, I maintain such a Google Alert on my name and that the article was first published online a month and a half ago. You’ve probably heard the famous adage, “I don’t care what the newspapers say about me as long as they spell my name right.” Well, Boon and Vohra didn’t even do me the courtesy of spelling my name right, which, fallible human that I am, I must admit annoys me. What annoys me a lot more, however, is the misstatement of our position. It is, I note, a very common straw man of our position, which Vohra and Boon double down on immediately after that passage:
The arguments put forth by Gorki [sic] and Novella[1] are based on the premise that our current understanding of basic human biology is sufficient to predict which therapies will work in humans. Interestingly, the hypothetical scenario used by these authors to help the reader understand the magnitude of the problem is the same strategy used by those trying to explain how little basic science research informs clinical therapies.[2] Unfortunately, it has become well known that basic science may or may not inform how therapies work in patients, more often ‘may not’.[3, 4] This is the reason given by pharmaceutical companies to justify why prescription medicines are so expensive.[5, 6] Since so many biologically plausible compounds ultimately fail to turn into viable therapies, the very few drugs that make it to market need to be priced in such a way as to compensate for all those that failed despite promising (and biologically plausible) laboratory findings.
Have Steve and I heard this argument before many times? Why yes. Yes we have. It’s a straw man so massive that, were it to be set on fire, it would be easily visible from the International Space Station. Yet proponents of unscientific medicine can’t seem to resist trotting it out, tarting it up, and sending it out to misrepresent our position. No one, least of all Steve and I, argues that we understand enough about basic human biology to say that knowledge is sufficient to predict what therapies will work in humans. What we did argue is that there is more than sufficient evidence from basic science, including physics, chemistry, and, yes, physiology to know that modalities like homeopathy and reiki can’t work. Let me just quote from our article for your edification. Because it’s behind a paywall, I will quote fairly liberally:
It should also be noted that ‘biologically plausible’ does not mean ‘knowing the exact mechanism’. What it does mean is that the mechanism should not be so scientifically implausible as to be reasonably considered impossible. In other words, the mechanism should not violate laws and theories in science that rest on far sturdier and longer established foundations than imperfect, bias-prone clinical trials. For example, homeopathy violates multiple laws of physics with its claims that dilution can make a homeopathic remedy stronger and that water can retain the ‘memory’ of substances with which it has been in contact before [9]. Thus, treatments like homeopathy should be dismissed as ineffective on basic scientific grounds alone. That is why we propose the term science-based medicine (SBM) as opposed to evidence-based medicine (EBM). SBM restores basic science considerations to EBM and is what EBM should be.
Later in the article, Steve and I argue:
Another pernicious effect of performing RCTs on CAM and IM modalities is that it leads to clinical trials of highly implausible treatments that may have a significant potential to cause active harm, as opposed to harm from substituting ineffective for effective treatment. One example is the Trial To Assess Chelation Therapy (TACT), in which chelation therapy was tested as a treatment in patients with heart disease: a US$30 million multicenter clinical trial initially funded by the National Center for Complementary and Alternative Medicine (NCCAM) with virtually no preclinical scientific basis, study sites at ‘alternative’ clinics where reliability was seriously questioned, and the potential for complications due to chelation of critical minerals [10]. A decade and tens of millions of dollars later, after failure to reach accrual goals, TACT results were reported [11,12] and were negative except for one subgroup (diabetics), for which there was ample reason to question the validity of the results [13]. Another example is the trial to test an ‘alternative’ treatment regimen for pancreatic cancer that involves extreme dietary modifications, juices, large quantities of supplements, and coffee enemas. After several years, abandonment of the RCT format for an unblinded ‘patient’s choice’ design, and considerable controversy over delays in publication, the results, when finally reported [14], were disturbing. One year survival of subjects undergoing this protocol was nearly fourfold worse than subjects receiving standard-of-care chemotherapy and worse than expected based on historical controls, all associated with poorer quality of life.
So it is very clear that we were not generalizing our arguments to claim that we know enough about human physiology to know in advance which treatments will work. That is a misstatement of our position that completely misses the point. Our argument says nothing about which treatments will work; it emphasizes predicting which treatments will not work, treatments for which doing clinical trials is a pointless waste of money that potentially endangers human subjects and whose results never convince proponents of alternative medicine to abandon treatments that don’t work. Our argument is that there are some modalities that are so inherently ridiculous, so completely without even a modicum of support, from a scientific standpoint that we already have enough data and knowledge to know that they can’t work (e.g., homeopathy and reiki, not to mention pretty much all “energy medicine”). Moreover, in specific cases that we enumerated testing such modalities can be viewed as unethical, as in the TACT (which we’ve discussed here at SBM many times before), trials of homeopathy for diarrheal diseases in Third World countries (also discussed here multiple times before); and testing a treatment like the Gonzalez protocol for pancreatic cancer (yes, also discussed multiple times here before).
The second part of our argument is that EBM was “blindsided” by CAM and as a result EBM has a “scientific blind spot” because an unspoken assumption behind the EBM evidence pyramid is that treatments do not make it to the stage of RCTs without having demonstrated compelling preclinical evidence of efficacy. Yes, we acknowledge that such evidence isn’t enough to predict which treatments will work, but, again, we never argued that it was. What we argue is that we already know enough about pseudoscientific modalities like homeopathy to know they can’t work. As Kimball Atwood put it, basic science or preliminary clinical studies provide evidence sufficient to refute some health claims (e.g., homeopathy and Laetrile), particularly those emanating from the social movement known by the euphemism “CAM.” Further, Atwood argued that EBM’s founders understood the correct role of the rigorous randomized controlled clinical trial: to be the final arbiter of any claim that had already demonstrated promise by all other criteria, including ‘basic science, animal studies, legitimate case series, small controlled trials, “expert opinion,” whatever (but not inexpert opinion).’ As he argued, EBM’s founders knew that such pieces of evidence, promising though they may be, are insufficient because they “routinely lead to false positive conclusions about efficacy.”
I often add to Dr. Atwood’s discussion an additional point emphasizing that determining the exact level of prior plausibility necessary to justify a RCT is a legitimate question. A 50% chance of a positive result? 10%? 1%? Less, even? Very likely the prior plausibility of all “complementary and alternative medicine” (CAM) treatments that are not supplements or herbal medicines falls into that very low range. The answer to that question will actually vary by disease and treatment, not to mention the expense of doing the trial. Thus, it comes down to more or less a value judgment in which preclinical estimates of prior probability influence the decision whether or not to proceed with a clinical trial, but factors of ethics, resources, and urgency of the question, play prominent roles in the decision as well. What I would hope that we all can agree on is that we shouldn’t be subjecting human subjects to tests of therapies whose prior plausibility is so low as to be indistinguishable from zero. Certainly, homeopathy, healing touch, and reiki, among others, fit this criterion quite well. Again, no one is arguing that basic science considerations can predict which therapies will work, although we do argue that in some cases basic science considerations can predict accurately which therapies will not work. I only repeat this again because it’s a message that critics like Vohra and Boon seem willfully unable to grasp and particularly prone to make a flaming straw man out of.
Now, remember the point we at SBM have been making about CAM advocates opting for less rigorous trials when more rigorous ones don’t give them the answers they want? Vohra and Boon demonstrate this very well:
Best available data suggest the majority of patients use complementary therapies.[11-13] Evidence about which therapies may be helpful, and which may be harmful, in whom, and why, is urgently needed to guide practice and policy. Randomised controlled trials have become the gold standard for evaluating treatment effectiveness, but they are expensive and resource-intensive. Rather than arguing that gathering clinical evidence about popular therapies is a folly, we suggest consideration of other kinds of innovative trial design, such as N-of-1 trials.[14] These trials can not only inform the care of a single patient in a cost-effective evidence-based manner, they may be aggregated to inform the care of a population.[15] Comparative effectiveness trials [16] or patient-centred effectiveness research are approaches that help to determine which interventions work best for specific patients in specific circumstances. A key component of this kind of research is the concept of pragmatic clinical trials that focus on assessing effectiveness (use of therapies in real-world settings) as opposed to efficacy[17] – the goal of the traditional RCT. There are an increasing number of rigorous study designs that should be used more to generate clinically useful knowledge and to highlight where it would be most useful to invest in additional mechanistic studies.
Not how Bohra and Boon start out with the CAM advocate’s favorite logical fallacy, argumentum ad populum (appeal to popularity) and then proceed to their second massive flaming straw man, the claim that we in any way said that “gathering clinical evidence about popular therapies is a folly.” (I mean, seriously, did these two even actually read our article?) One more time: No one, least of all Steve and I, has said that “gathering clinical evidence about popular therapies is a folly.” What we have said is that doing RCTs on magic (like homeopathy, reiki and “energy medicine,” and treatments like chelation therapy for cardiovascular disease or the Gonzalez protocol for pancreatic cancer) is a waste of resources that can endanger patients and that negative results of these RCTs fail to persuade CAM practitioners to abandon pseudoscientific treatments. As for “pragmatic” trials, these are trials that are done to determine how effective modalities that have already been demonstrated to be efficacious in RCTs are in the “real world.” They are not intended for treatments whose efficacy has not yet determined. Using pragmatic trials as though they are efficacy trials is a strategy bound to result in false positives, because pragmatic trials usually have no placebo control group and sometimes no control group at all. It’s with good reason that Steve has referred to the use of pragmatic trials in CAM as “bait and switch” and Harriet Hall as referred to them as “Cinderella medicine.” (This is a known problem with pragmatic trials that isn’t limited to CAM, too.) No wonder Andrew Weil likes pragmatic studies so much!
Finally, no one here is arguing against patient-centered effectiveness research or comparative effectiveness research. However, once again, before we can compare the effectiveness of different treatments, we have to know that each of the treatment modalities being compared is actually efficacious; i.e., that there is actual effectiveness in each of the modalities to be compared and provide meaningful results. As for the “N-of-1” trial, homeopaths tend to like to propose such trials because to them “N-of-1” trials resemble homeopathic provings. Be that as it may, conducting N-of-1 trials is anything but trivial. Moreover, as the U.S. Agency for Healthcare Research and Quality puts it:
In parallel group RCTs, blinding of patients, clinicians, and outcomes assessors (“triple blinding”) is considered good research practice. These trials aim to generate generalizable knowledge about the effects of treatment in a population. In drug and device trials, the consensus is that it is critical to separate the biological activity of the treatment from nonspecific (placebo) effects. (For a broader view, see Benedetti et al.26) In n-of-1 trials, the primary aim is usually different. Patients and clinicians participating in n-of-1 trials are likely interested in the net benefits of treatment overall, including both specific and nonspecific effects. Therefore blinding may be less critical in this context. Nevertheless, expert opinion tends to favor blinding in n-of-1 trials whenever feasible.
See why CAM advocates love N-of-1 trials? After all, nonspecific effects are pretty much the main reason why patients and clinicians think that, for example, acupuncture works for anything.
Vohra and Boon then conclude:
We should encourage and celebrate basic science and discovery research as a way to learn about our universe, including the human body. We urge researchers to update their thinking: translational research should not be limited to one direction – from basic science to bedside applications. Complementary therapies are used by the majority of the population. There is a need to explore therapies that appear to offer clinically relevant benefit or harm by investigating their mechanism of action through basic research (i.e. bedside to bench). Basic science is clearly needed, but it should never be a reason to deny the opportunity to evaluate clinical therapies to assess their safety and effectiveness. Both types of research are informative, both advance the human condition, and both have their weaknesses. Together they can be immensely powerful.
This paragraph is just plain ridiculous. First of all, we never said that translational research should be one direction; so there’s the third major flaming straw man. In fact, we explicitly noted that it goes both ways in our paper. Second, complementary therapies are only used by the “majority of the population” if you include exercise, spirituality, vitamins, and diet (among other things) as being “complementary.” It’s a fallacy that we at SBM have discussed many times before, wherein modalities like diet and exercise are the “Trojan horse” that lets the pseudoscience past the walls of academia. I would, however, argue that Vohra and Boon are just plain wrong to argue that basic science “should never be a reason to deny the opportunity to evaluate clinical therapies to assess their safety and effectiveness.” If they don’t believe me, then I would invite them to participate in a double blind, randomized controlled trial of parachute usage to decrease the morbidity and mortality due to jumping out of airplanes. After all, if basic science should never be used as a reason to deny the opportunity to evaluate clinical therapies, then the basic science (physics) that says that a human body falling hundreds or thousands of feet at terminal velocity will go splat when it hits the unyielding earth and that this splatting will result in death or extreme injury—the basic science of biology, people!—shouldn’t be a consideration in assessing the effectiveness of parachute usage in preventing death from falls from high places.
A less extreme example would be testing intercessory prayer for cardiac conditions. Oh, wait. That’s been done. Never mind. That’s the problem with testing highly implausible claims with no physical basis in reality.
Like homeopathy for anything.
Homeopathy for ADHD?
Of course, Steve and I have to wonder why Vohra and Boon were so unhappy with our little editorial, which was only around 1,500 words. (Yes, Steve had a salutary effect on my brevity.) It turns out that we’ve met both Heather Boon and Sunita Vohra before. For example, Jann Bellamy and a certain “friend” of the blog have both taken her to task for advocating the use of pseudoscientific CAM treatments in children. It turns out that Vohra is Centennial Professor in Department of Pediatrics, Faculty of Medicine and Dentistry at the University of Alberta who holds a joint appointment in the School of Public Health. She appears to have started out evidence- and science-based but then, as too often happens, drifted into CAM, becoming:
…founding director of Canada’s first academic pediatric integrative medicine program; founding director of first pediatric integrative medicine fellowship program in North America; founding director of the world’s largest pediatric Complementary and Alternative Medicine (CAM) network (www.pedcam.ca); she advises Health Canada (Pediatric Expert Advisory Committee); led the University of Alberta’s membership in Consortium of Academic Health Centers for Integrative Medicine; is past Chair of the American Academy of Pediatrics Section on Integrative Medicine (2012-2014). Dr. Vohra was recognized for excellence in CAM research (Dr. Rogers Prize $250,000) and inducted as fellow of Canadian Academy of Health Sciences, one of the highest honours for any member of the Canadian health sciences community.
I suppose Steve and I should be honored to have attracted the attention of such a heavy hitter in the world of CAM. Based on her stature in CAM, you’d think her arguments would be better, but, as I demonstrated above, you’d be wrong.
I’m not so much interested in Dr. Vohra, though. Rather, I’m more interested in Heather Boon, because she’s been in the news in Canada lately regarding homeopathy. Also, Vohra and Boon have collaborated on clinical trials of homeopathy in children before. No wonder our little “Science and Society” editorial hit a nerve!
In any case, Heather Boon is a pharmacist who is Professor and the Dean for the Leslie Dan Faculty of Pharmacy, University of Toronto. As our own Scott Gavura discussed nearly two weeks ago, she is also the principal investigator of a study of homeopathy that’s drawn criticism from scientists all over the world:
A University of Toronto study on homeopathic treatment for children with Attention Deficit Hyperactivity Disorder is being heavily criticized by scientists who claim it legitimizes a pseudoscience.
Two Nobel laureates are among 90 scientists from universities around the world who have signed an open letter calling the clinical trial into question.
“We are curious about why, given the need to investigate natural therapies that may actually have a potential for benefit, and saddled with a scarcity in funding, a Department of Pharmacy is interested in investigating a subject that has been … found wanting both in evidence and plausibility,” reads the letter addressed to Heather Boon, dean of the U of T’s Faculty of Pharmacy, who is leading the study.
Indeed.
You would think that a pharmacist would know why homeopathy is pseudoscience, but apparently Dr. Boon does not. (If she did, she wouldn’t have written the criticism of the article by Dr. Novella and myself that she did; on the other hand, she has been critical of homeopathy in the past, which makes her criticism of our article and her spearheading this trial particularly odd) In particular, Joe Schwarcz, a chemist at McGill University and friend of the blog, has been particularly scathing in his criticism, noting that:
The study is actually to be carried out at the Riverdale Homeopathic Clinic, a private institution that also offers ear candling, cranial sacral therapy and “nosodes,” which are homeopathic versions of vaccines. No public funding is involved; support comes from a foundation dedicated to alternative medicine.
Me being me and all that and in particular me being about the details of the clinical trial, I looked up Boon’s trial on ClinicalTrials.gov (identifier NCT02086864). It’s a clinical trial of homeopathy in ADHD, a developmental disorder characterized by developmentally inappropriate levels of inattention and/or hyperactive-impulsive behavior with significant impairment in at least two settings. Boon then argues that homeopathy “has been shown to be a promising intervention for ADHD” as justification of this trial, whose three aims are:
     1.  To determine if there are any specific effects of homeopathic medicines in the treatment of ADHD
     2.  To determine if there any specific effects the homeopathic consultation alone in the treatment of ADHD
     3.  To determine if there is an overall effect of homeopathic treatment (homeopathic medicines plus consultation) in the treatment of ADHD.
There are three groups:
    • Arm 1 (Verum group): a treatment arm where the participant will receive homeopathic consultation plus a homeopathic remedy
    • Arm 2 (Placebo group): a treatment arm where the participant will receive homeopathic consultation plus a placebo remedy
    • Arm 3 (No treatment/Usual care group): a wait list arm where the participant will not receive homeopathic treatment as part of the study.
The pilot trial to which the study justification refers is this pilot trial published last year (NCT01141634), which was an open label trial without a control group. Not surprisingly given that it was an open-label uncontrolled pilot study, this study was reported as positive, as judged by changes between the pre- and post-study Conners Global Index – Parent (CGI-P) T-score, a measure of ADHD severity.
In fact, I bet I can predict how Boon’s current study will most likely turn out. There will be no specific effect due to any homeopathic treatment, given that anything diluted more than about 11C or 12C has been diluted to the point where no drug is left and will therefore be water. Indeed, that is why homeopathy is pseudoscience. In addition, there might be an improvement based on the “personalized homeopathic consultation.” In other words, results observed in subjects in Arms 1 and 2 will likely be indistinguishable statistically from each other, and results in those arms will likely be better than the results in Arm 3. As I so frequently say in studies like this, there’s no need for the pseudoscience; the same sort of study could be done using standard treatment to see if longer, more personalized consultations produces a better result, no magic needed. In brief, this entire study is a pseudoscientific waste of time.
So how did Boon respond? She wrote a special article that was published in The Montreal Gazette in response to Prof. Schwarcz’s column, entitled “Why would anyone think a rigorous clinical trial is a bad idea?” It reads like a rehash of the article published in FACT, just directed at a lay audience. The question itself that is the title, reiterated near the beginning of the article, shows just how little Boon understands.
She is also, as they say, talking out of both sides of her mouth. She starts out asserting:
Our study — one of the most rigorous studies of homoeopathy to date — is being conducted because patients are using homoeopathy to treat ADHD. Previous studies on this topic have produced varying results and have been criticized for methodological flaws.
We are studying this topic for the same reason most scientists conduct studies: to discover the truth, or at least to get closer to it. As a scientist, I am committed to conducting high-quality studies that help patients and clinicians make evidence-informed choices about a wide range of popular products and therapies, and this study is no different.
She even boasts that her study had received approval from Health Canada and two Research Ethics Review Boards while expressing puzzlement how a “well-designed clinical trial legitimizes anything, as we don’t know what the study will find.” Again, Boon completely misses the point, just as she did in her criticism of our article. Her clinical trial is studying pixie dust. It’s tooth fairy science, as it is studying the efficacy of a treatment whose very premises are based on vitalistic magic. There is no scientific reason to think that homeopathy will benefit ADHD. It is water.
She then justifies her choice:
Biological plausibility is one way to choose which interventions to study, and is the most common way we bring new pharmaceutical drugs to market. Given the number of promising drug candidates that never make it to market, it is clear that biological plausibility does not always translate into efficacy or effectiveness in patients. Another way to identify interventions to study is to start with an observation — often claims from patients that a product, treatment or therapy helps them — and then set out to investigate this observation. In either approach, determining whether something ultimately “works” or not is achieved through investigation in rigorously designed clinical trials.
Again, no one is arguing that you always have to start with preclinical observations. In fact, the paradigm is often “bench to bedside to bench” or “bedside to bench to bedside.” As Steve and I pointed out in our article, clinical observations and lab observations frequently cross-pollinate. The idea is that if there is a clinical observation of an effect, then the first course is to go to the lab and try to figure out why that might be. Of course, for such an approach to work, regardless of what comes first, there has to be a convincing clinical observation, and certainly Boon and Vohra’s pilot study of homeopathy for ADHD does not qualify, nor do any studies of homeopathy, as even the “positive” ones tend to be just barely “statistically significant,” poorly designed, or fraught with potential bias. In other words, when it comes to homeopathy there’s no “there” there. For a clinical observation to have even a hope of trumping the nearly two centuries’ worth of basic science that tells us homeopathy is impossible would require unequivocal, undeniable observations on multiple fronts.
Oddly enough, Boon then backtracks a little, as if to say, “Oops! We’re not trying to determine if homeopathy works after all!” See what I mean:
Our pilot study was designed to tell us about patients’ experiences using homeopathic therapies to treat ADHD, not to assess the efficacy of those treatments. Many patients in this study felt better after visiting a homeopath. Of course, there are many possible explanations for this, which is why we have embarked on another, more comprehensive study to investigate two possible explanations. Employing a randomized, placebo-controlled three-arm study, we will be able to identify whether or not patients in the trial have a decrease in symptoms after homeopathic treatment. If patients’ symptoms decrease, we will be able to explore whether this effect appears to be related to the consultation with the homeopath or to the homeopathic medicine or a combination of both.
Except that, as I mentioned above, you don’t need to have a pseudoscientific remedy like homeopathy to do a study like this. Just do it with standard care for ADHD. It’s unethical to use a pseudoscientific and ineffective treatment like homeopathy in a clinical trial, as the most recent iteration of the Declaration of Helsinki points out, stating that “medical research involving human subjects must conform to generally accepted scientific principles, be based on a thorough knowledge of the scientific literature, other relevant sources of information, and adequate laboratory and, as appropriate, animal experimentation.” Unfortunately, ethics boards never seem to understand that homeopathy conforms to none of these things.
Boon concludes with a particularly nauseatingly sanctimonious, self-righteous rhetorical question:
We are putting homeopathy to the test and trusting in the scientific method to help us in our search for the truth.
After all, isn’t that the purpose of research?
My retort: We have already trusted in the scientific method that tells us that, for homeopathy to work, several laws of physics would have to be not just wrong, but spectacularly wrong. That scientific method tells us that there’s no point in subjecting children with ADHD to homeopathy. As scientists, we always acknowledge that there is a tiny chance those mountains of research in physics, chemistry, and biology that tell us homeopathy is impossible could be mistaken. However, for homeopathy to be shown to have plausibility distinct from zero would require basic science in favor of the principles of homeopathy on the order of quality and quantity of research since the time of Amedeo Avogadro, whose number demonstrated that homeopathy is bunk. That research doesn’t exist. Respecting science doesn’t mean subjecting human subjects in clinical trials to quackery like homeopathy in the futile hope that it will work when the research to demonstrate that homeopathy is pseudoscience has existed for over 200 years and been accepted for at least 150 years.
In reality, it is not Boon who respects science. By embracing homeopathy as a legitimate research topic, she has embraced pseudoscience. It is Joe Schwarcz and the scientists and physicians who signed his letter who do.
ADDENDUM: Steve Novella has also posted a rebuttal to Vohra and Boon entitled “Basic Science Should Inform Clinical Science.”

Answering Our Critics, Part 1 of 2
Harriet Hall on September 24, 2013
Link
Some people don’t like what we have to say on Science-Based Medicine. Some attack specific points while others attack our whole approach. Every mention of complementary and alternative medicine (CAM) elicits protests in the Comments section from “true believer” users and practitioners of CAM. Every mention of a treatment that has been disproven or has not been properly tested elicits testimonials from people who claim to have experienced miraculous benefits from that treatment. In previous articles I have compiled the criticisms of what I wrote about Protandim and Isagenix. It’s instructive to read through them. We welcome rational and substantive criticism, but most of these comments are neither.
Our critics keep bringing up the same old memes, and it occurred to me that rather than try to answer them each time, it might be useful to list those criticisms and answer them here. In future, when the same points are raised, we could save time and effort by linking to this page and citing the reference number. I know this list is not comprehensive, and I hope our readers will point out anything I’ve omitted. Here are some of the criticisms we keep hearing:
1. Big Pharma is paying you to promote their products and discredit CAM.
No it isn’t. We are not Pharma shills. We are not paid anything for writing this blog. We do not get money from pharmaceutical companies. We do not accept gifts from drug companies. We do not get kickbacks for prescribing certain drugs. We have no incentive to favor drugs over other treatments. Incidentally, critics who prefer natural remedies to pharmaceuticals should note that many CAM diet supplements are sold by subsidiaries of Big Pharma.
2. You’re biased.
Yes, we are, and that’s a good thing. We are biased in favor of science and reason. We are biased against claims that have been tested and disproven and that are incompatible with the rest of scientific knowledge. We are biased against health care providers telling patients things that are not true, presenting opinions as if they were facts. We are biased against using placebos because we consider it unethical. We are not biased against any CAM treatment just because it is CAM; we contend that there is only one medicine, that treatments have either been proven to work or they haven’t, and that all claims should be held to the same standard and tested by the same scientific method.
3. You’re afraid of the competition.
Not at all. The SBM bloggers aren’t affected by “competition,” since we are either salaried or retired and don’t make more money by seeing more patients. Most doctors are overworked and already have all the patients they can handle. CAM has only a very small share of the healthcare market. It’s not that we are afraid CAM will take patients away from us, it’s that we are concerned for our patients’ welfare and don’t like to see them lied to, given ineffective treatments, persuaded to reject effective treatments, and sometimes harmed.
4. Science isn’t everything: there are other ways of knowing.
When it comes to knowing whether a treatment is effective, there is only one reliable way of knowing: controlled testing using the scientific method. Intuition, tradition, revelation, “stoned thinking” à la Andrew Weil, dreams, extrapolation, speculation, and personal anecdotal experience can lead people to strong beliefs, but we can’t trust those beliefs to reflect reality. Only the scientific method can give us reliable knowledge. No matter how convincing they sound, claims must be tested before we can assume they are true.
5. It worked for me.
Maybe, maybe not. You can only know that you improved after the treatment; you can’t know for sure that you improved because of the treatment. That could be a post hoc ergo propter hoc logical fallacy. You may not be able to imagine any other possible explanation, but that doesn’t mean there isn’t one. Barry Beyerstein has explained some of the many ways people come to believe that a bogus therapy works. Also see the Quackwatch article on spontaneous remission and placebos.
6. Try it yourself.
Trying it for yourself is not a reliable way to find out if a treatment works. If the symptoms resolve, you have no way of knowing whether they resolved due to the treatment or whether they would have gone away anyway without treatment, or whether some other factor caused the improvement. That’s why science uses control groups. And what if your symptoms don’t resolve? That doesn’t rule out the possibility that the treatment works for 99% of patients and you just happened to be one of the other 1%. If you try a remedy and get better, it’s reasonable on a practical basis to try it the next time you have the symptoms, but it’s not acceptable to cite your experience as proof that “it works.”
7. Huge numbers of people use X, and they couldn’t all be wrong.
Yes, they could. Popularity is no indication of truth. Over the centuries, how many people have believed in astrology? For centuries, everyone believed bloodletting was effective in balancing the humors to treat disease. Only when it was properly tested did doctors discover they’d been killing patients instead of helping them.
8. It’s been used for centuries.
This is the argument from antiquity and is a logical fallacy. Dr. Novella explains why. It could be ancient wisdom, but it could just as well be ancient error carried over from a prescientific era.
9. It’s natural, therefore it’s safe.
Not necessarily. Many natural substances are poisons. Any natural remedy must be tested for efficacy and safety by the same standards we use to test “unnatural” remedies like pharmaceuticals. Dr. Novella has explained the naturalistic fallacy here. Herbs are drugs too, and anything that has an effect can have a side effect.
10. There is proof that X is correlated with Y (cites study).
Correlation does not prove causation. The rise in the number of diagnoses of autism correlates almost perfectly with the rise in the sales of organic food, but that doesn’t mean organic food causes autism. Correlation can be due to chance, error, poor data collection, and many other things. There may not really be a true correlation; and even if there is, that doesn’t tell us whether X caused Y or Y caused X or whether X and Y might both be caused by Z.
11. There are hundreds of studies that show X works.
In all-too-many cases, most of the studies supporters cite are in animals or test tubes, and others are opinion pieces, speculations, and irrelevant studies. There may be other, better quality human clinical studies that show it doesn’t work. Studies can be found to support almost any claim. Half of all published studies are wrong, for a variety of reasons that are constantly discussed on this blog. You can’t just look at positive studies: you have to look at the entire body of published evidence. That’s where systematic analyses come in. And they may not reflect reality: there may be negative studies that we don’t know about because they were never published: the file drawer effect and publication bias. And remember Carl Sagan: “Extraordinary claims require extraordinary evidence.” It would take a great deal of evidence indeed to overthrow all the established science that tells us homeopathy can’t possibly work as advertised.
12. You are just robotically supporting the official party line of mainstream medicine.
Most of the time we come to the same conclusions as the majority of mainstream doctors, because we are looking at the same evidence. When a body of experts evaluates all the published studies and makes evidence-based recommendations, we usually agree with them. Sometimes we disagree with their interpretation of the evidence (especially if they fail to consider prior probability or are including Tooth Fairy science) or with the way they have stated their conclusions. We understand that evidence-based guidelines are only guidelines, and judgment should be used in applying them to individual patients. We don’t agree just because we consider them authorities. There is a difference between the appeal to authority (he’s a professor at Harvard, so we should believe everything he says) and accepting the consensus of experts who know more about the field that we do (if 10 top mechanics all agree that your carburetor needs replacing, it is only reasonable to assume that it really needs replacing). All too often, CAM advocates are the ones who are parroting unreliable “authorities” who don’t know what they’re talking about.
13. Doctors only treat symptoms, not the underlying cause of disease.
This stupid CAM mantra is a vile, false accusation. Doctors treat the underlying cause whenever possible. If a patient has pneumonia, we don’t just treat the fever, pain, and cough; we figure out what microbe is responsible and provide the appropriate antibiotic. If a broken bone is painful, we don’t just treat the pain, we immobilize the fracture so it can heal. If a patient is in agony from pain in the right lower quadrant of the abdomen, we don’t just treat the pain, we try to figure out if the underlying cause is appendicitis, and if it is, we operate.
The people who accuse us of not treating “the underlying cause” are often the ones who think all disease is due to one bogus underlying cause (subluxations, disturbances of qi, poor diet, etc.). They also tend to use a single treatment (when all you have is a hammer, everything looks like a nail). I once googled for “the one true cause of all disease” and found 63 of them. Science-based medicine recognizes 9 whole categories of causes, with the mnemonic VINDICATE.
14. Science-based medicine can’t explain why some people get a disease and others don’t.
Neither can CAM. But doctors do have some pretty good ideas why it happens: exposure to infections, number of organisms that get into the body, genetic factors, toxins, immune deficiency, chance, and so on. CAM claims to fully understand why it happens, attributing it to some single cause that impairs optimum health (like a subluxation or a disturbance in qi, or improper diet). But they have not been able to show they understand the answer to that question any better than conventional medicine does, or that their understanding leads to better patient outcomes.
15. Conventional medicine kills patients.
I wrote about the “Death by Medicine” meme here. Critics gleefully cite statistics for drug reactions, medical errors, and iatrogenic deaths; but it is irrational to look at those numbers in isolation. Harms must be weighed against benefits. Medicine saves far more people than it kills. Many of those who develop treatment complications would have died even sooner without treatment. All effective treatments have side effects. We look at the risk/benefit ratios and reject treatments where the risk is greater than the potential benefit. The risk/benefit ratio of CAM should be compared to that of conventional medicine; if there is no proven benefit, no degree of risk can be justified.
16. Your minds are closed.
We are open to any new treatment, no matter how implausible, if only it can be shown to be safe and effective in well-designed controlled trials. Before we can ask how it works, we must ask if it works. If homeopathy had shown the same spectacular degree of success as penicillin, everyone would be using it. When Helicobacter was proposed as the cause of ulcers, it only took a few years for the evidence to accumulate and for antibiotics to become the treatment of choice. When a treatment like acupuncture has been studied for decades and even for centuries and its effectiveness is still uncertain, it is only reasonable to stop studying it and spend our research money elsewhere. We don’t need to keep an open mind about perpetual motion or a flat earth, and we don’t need to keep an open mind about homeopathy.
17. You are too prejudiced against CAM to look objectively at our evidence for it. No amount of evidence would change your minds.
We change our minds all the time based on changing evidence. We look at the best evidence for a CAM treatment before we reject it. We would accept CAM if it could present the same quality and quantity of evidence that it takes to reach a scientific consensus about any other medical treatment. What would it take for CAM advocates to change their minds? Most of them hold their beliefs so firmly that they reject any evidence to the contrary. One practitioner told me he would keep using his pet method even if it was definitely proven not to work, because “his patients liked it.”
18. Science keeps changing its mind.
Yes, and that’s a good thing. Scientific conclusions are always provisional. We follow the evidence wherever it leads, and we often have to change course as new evidence becomes available. CAM refuses to change its mind even in the face of clear evidence. Scientific medicine stops using treatments if they are proven not to work. CAM never rejects any treatment, and hardly ever tests one of its treatments against another to see which is superior.
19. Doctors are only out to make money.
I think most doctors go into medicine not because they want to get rich but because they want to help people. Medical education is long, grueling, and expensive. Most doctors incur substantial debts for their education and need years to repay them. The nice houses and cars don’t come until long after graduation. The median net worth for physician households is $700,000 and their median income is going down. The ones who really get rich are those who market bogus remedies or spread misinformation (like Dr. Oz, Andrew Weil, Burzynski, Daniel Amen, Kevin Trudeau, and all the companies that sell diet supplements and miracle weight loss aids). Boiron sold 566 million Euros worth of homeopathic remedies in 2012.
20. Alternative treatments are individualized and can’t be subjected to the same tests as pharmaceuticals.
Anything can and should be tested by scientific methods. For instance, homeopaths could prescribe individually in whatever way they chose, then the remedies they prescribe could be randomized with placebo controls and dispensed by someone else with double-blinding. Or the objective outcomes of treatment by conventional vs. CAM providers could be compared.
21. Doctors don’t do prevention.
They most certainly do! Who do you think invented vaccinations and preventive screening tests? Don’t you know about the US Preventive Services Task Force? MDs routinely talk to patients about weight control, diet, seatbelts and other safety topics, alcohol, drugs, domestic violence, exercise, etc. Studies on these topics are constantly appearing in the major medical journals. And there’s no evidence that the preventive efforts of CAM providers result in any better health outcomes than those of MDs.
22. Doctors don’t know anything about nutrition.
They understand the science of nutrition and advise their patients based on the available scientific evidence. Even if they haven’t taken a specific course titled “nutrition,” they learn how vitamins, minerals, and other nutrients are utilized biochemically throughout the body. CAM providers claim to know more about nutrition, but they often give pseudoscientific or unfounded diet advice.
23. CAM is better because it’s holistic.
CAM appropriated the holistic principle from mainstream medicine. Doctors are taught holistic principles in medical school. We are taught that the secret of the care of the patient is caring for the patient, not just treating the disease. Part of the standard medical history is a “social history.” Good clinicians consider the patient’s family, lifestyle, job, stresses, education, diet, socioeconomic status, beliefs, and everything about the individual that might have an impact on medical care.
24. We don’t need studies; we have plenty of testimonials.
10 anecdotes are no better than one; 100 are no better than 10. Anecdotal evidence is unreliable, no matter how many anecdotes you have accumulated. This lesson has had to be relearned over and over again throughout the history of medicine. Just think of how many testimonials there were for bloodletting in the Middle Ages. Anecdotes are useful, but only as a guide to what to investigate with scientific studies.
25. Why won’t you believe us?
We do. We believe you believe what you are telling us. We believe you had the experience you related. But that doesn’t mean your interpretation of your experience is true.
26. If you think X doesn’t work, why don’t you do a study to prove it?
It’s not that we think X doesn’t work, it’s that there is no evidence to make us think it does work. It is not up to us to prove a negative. It is up to the person making the claim to provide the supporting evidence. If I told you that putting a poker chip in your gas tank would give you better mileage, you should ask me to prove it. You shouldn’t feel obligated to either put a poker chip in your tank or do a study to prove it didn’t work.
27. Natural remedies don’t get tested because they can’t be patented and there’s no profit in it.
Nonsense. Many prescription drugs were developed from plants. The plant itself can’t be patented, but the drug company can isolate the active ingredient and patent that, or even improve on it with a synthetic version that is more effective and has fewer side effects. They can patent a unique method of converting a plant into a pill. There’s plenty of money to be made in herbal medicines, diet supplements, and even plain old vitamins: they generate billions of dollars’ worth of profits every year.
28. The medical establishment would drum out any doctor who tried to publish studies going against the party line, showing that X worked or that condition Y was real.
Quite the contrary. Peer review would critique the study. If it was a good study, others would investigate. A doctor who discovered a new disease or treatment would be honored. The treatment of ulcers with antibiotics is a case in point: Drs. Marshall and Warren won a Nobel Prize for their discovery. Montagnier was awarded a Nobel Prize for discovering the virus that causes AIDS only two years after the first reports of “gay-related immune deficiency syndrome.” Real diseases and new treatments are quickly recognized by the medical community.
29. You can’t know about it if you haven’t experienced it.
You don’t have to have been bitten by a snake to know how to treat snakebite. Male obstetricians are proof that you can deliver babies without having been pregnant yourself. We can know that antibiotics work for pneumonia without having had pneumonia ourselves. You don’t have to have used Perkins tractors to know that they don’t work. In fact, personal experience is a handicap: it tends to interfere with the ability to objectively assess the evidence.
And finally:
30. If CAM makes people feel better, why deny them that? Even if it’s just a placebo, isn’t that a good thing?
That merits its own post, which will appear as Part 2 next week.
Note: If readers can think of other recurring memes that I’ve omitted, I can add them to next week’s post if you let me know in time.

Answering Our Critics, Part 2 of 2: What’s the Harm?
Harriet Hall on October 1, 2013
Link
Last week I posted a list of 30 rebuttals to many of the recurrent criticisms that are made by people who don’t like what we say on SBM. I thought #30 deserved its own post; this is it. At the end, I’ve added a few items to the original list.
What’s the harm in people trying CAM? Science-based medicine has been criticized for being too rigid and intolerant. Why do we insist on randomized placebo-controlled trials to prove that a treatment is safe and effective? Isn’t it enough that patients tell us they feel better? Isn’t that what we all want, for our patients to feel better? Even if the treatment only works as a placebo, isn’t that a good thing? What’s the harm in that?
The albuterol/placebo study
I would argue that we don’t just want our patients to think they are better, we want them to actually be better. A study that illustrates that principle has been discussed on this blog before, here and here.
A group of patients using an effective albuterol asthma inhaler was compared to 2 placebo groups (a placebo inhaler group and a sham acupuncture group), and to a group that got no treatment at all. Patients reported the same relief of symptoms with each of the two placebo controls as with the albuterol inhaler; all three groups reported feeling significantly better than the no-treatment group. It could be argued that placebos are an effective treatment for the subjective symptoms of asthma.
But when they used an objective measure, improvement in lung function, airflow only increased in the albuterol group. The placebo groups were indistinguishable from the no-treatment group. So here you have patients who feel better and think they are better but who aren’t actually better, who might not recognize the severity of an asthma attack in time to get to the ER and avoid a fatal outcome. As Peter Lipson said, we have effective, lifesaving treatments for asthma, and treating an asthma attack with a placebo is folly.
An accompanying editorial was very misguided. It said:
What is the more important outcome in medicine: the objective or the subjective, the doctor’s or the patient’s perception? This distinction is important, since it should direct us as to when patient-centered versus doctor-directed care should take place… For subjective and functional conditions — for example, migraine, schizophrenia, back pain, depression, asthma, post-traumatic stress disorder, neurologic disorders such as Parkinson’s disease, inflammatory bowel disease and many other autoimmune disorders, any condition defined by symptoms, and anything idiopathic — a patient-centered approach requires that patient-preferred outcomes trump the judgment of the physician.
Should the patient run the show?
Some of those conditions are not just defined by symptoms: they also have physical findings and laboratory abnormalities. Admittedly, only the patient can tell us whether he is experiencing pain, depression, etc. and we should always listen and believe him. But things like changes in behavior, medication use, and Parkinsonian tremor can be observed by others, and autoimmune disorders can be followed with blood tests.
Should patient-preferred outcomes be our goal? Do we really want to offer any treatment that makes the patient feel better? Penn Jillette famously said, “If all you want is to feel better, take heroin.” What if you carry the patient-centered concept to its extreme? What if a patient prefers high doses of heroin for headaches, or antibiotics for a cold, or demands unnecessary surgery? Apotemnophiliacs will be happy with the outcome if a normal, healthy limb is amputated. Heroin makes people feel euphoric, patients who get a prescription for antibiotics feel better than if they become angry because their wishes are thwarted, and a patient might feel reassured if an unnecessary exploratory laparotomy finds nothing wrong. Should that trump the judgment of the physician? I think not!
Doctors are consulted because they have expert knowledge. They have a responsibility to educate and advise patients and to work with them, taking their personal preferences into account, to do what’s in their best interest, not just what they think they want. Medicine is not like a retail store where the customer is always right. Placebos are unethical, and deception tends to undermine the doctor/patient relationship.
Just how powerful is the placebo?
The placebo has been highly over-rated. One writer even said it was proof that God exists. It has been claimed that one-third of all illnesses can be cured with placebo, but that’s not true. It’s a misunderstanding of a 1955 study by Henry Beecher. He looked at studies with a placebo arm and found that on average, one-third of subjects in the placebo arm reported improvement. But those subjects were not all experiencing placebo effects; some of them would have reported improvement with no treatment, due to the natural course of the disease or other factors.
Two later studies by Hrobjartsson and Gotzsche in 2001 and 2010 looked at studies that included both a placebo arm and a no treatment arm and concluded that:
We did not find that placebo interventions have important clinical effects in general. However, in certain settings placebo interventions can influence patient-reported outcomes, especially pain and nausea, though it is difficult to distinguish patient-reported effects of placebo from biased reporting. The effect on pain varied, even among trials with low risk of bias, from negligible to clinically important. Variations in the effect of placebo were partly explained by variations in how trials were conducted and how patients were informed.
In their opinion, there is no place for the use of placebos outside of clinical research studies. Placebos may relieve headaches, but they can’t cure cancer or pneumonia. Antibiotics work just as well if the patient is asleep or in a coma; placebos only work if the patient is awake and knows he has been given something.
There’s not just one placebo effect, but a number of different placebo responses that work by different mechanisms including anxiety reduction, expectation, activation of the brain’s reward centers, conditioning, and learning. Fabrizio Benedetti has been doing exciting research on placebos, studying the physiological changes that occur when someone responds to a placebo. He also believes there is no place for placebos in clinical practice.
The stages in CAM reasoning
CAM advocates tend to pass through these stages:
     1.  They accept testimonial evidence and see no need for scientific testing.
     2.  They recognize that the scientific community will not accept their claims without scientific testing.
     3.  They do some studies.
     4.  When flaws in the studies are pointed out, they try to do better studies.
     5.  A systematic analysis of all studies fails to support their claims.
     6.  They are forced to admit that there is no convincing scientific evidence that their treatment works better than placebo.
     7.  They argue that even if the treatment is only a placebo, it should be used for its placebo benefit.
     8.  They argue that placebos are “effective” and that it is acceptable to lie to patients.
What’s the harm?
Val Jones coined the word “shruggie” to denote doctors who think alternative medicine is harmless and is not worth fighting. We think the “shruggies” are wrong.
Is there any harm in telling a patient that an ineffective treatment has real objective benefits? I would argue that this constitutes lying, is unethical, and undermines the doctor/patient relationship. If a patient finds out that the doctor has fooled him with a sugar pill, he won’t be able to trust the doctor again.
What if the patient is already using an ineffective treatment and believes it is helping him? We should ask if it is exposing the patient to a significant likelihood of harm. We should ask if the patient is using it instead of other treatments that have been proven to work or that might be lifesaving. If the answers to both questions are “no,” it is kinder and more humane not to challenge the patient’s belief.
We should have firm rules about evidence-based medicine and ethical principles, but there are times when the rules should be broken in the name of common sense and empathy. What if an 80-year-old woman has been getting monthly B12 shots for 40 years and is convinced she needs them, even though science tells us there is absolutely no legitimate medical indication for her to get them? It’s very unlikely that we could say anything that would change her belief. Challenging her belief will only cause her distress. If we deny her request, she will get the shots elsewhere. It’s not likely that the shots will harm her, and getting the shots is comforting to her and makes her feel better. In that case, I would break the rules and let her have the shots. I would feel a small twinge of guilt for breaking the rules, and we should feel that guilt whenever we break the rules, to minimize the chances of getting on a slippery slope and breaking them when we shouldn’t. We should break them only with very good reasons with which other reasonable people would likely agree.
So in individual, select cases, belief in ineffective treatments is relatively harmless and need not be confronted. But in general, false beliefs can do a lot of harm. There is a whole website, What’s the Harm, devoted to the harm that false beliefs can do. So far, it has compiled reports of 368,379 people killed, 306,096 injured and over $2,815,931,000 in economic damages. Please go there and look at some of the accounts of patients who have died or suffered serious consequences because of false medical beliefs about everything from acupuncture to vitamin megadoses.
The concept of comfort measures
What does “effective” mean? It’s important to understand the difference between objective outcomes and patient perceptions. We can conclude from the evidence that acupuncture is merely a theatrical placebo. We can conclude that it would be unethical for us to recommend it. But if a patient is already using acupuncture and feels it is effective in relieving his symptoms, that falls into the category of comfort measures, where the patient is deriving a degree of comfort from a procedure with no objective effects. Things like backrubs, fluffing pillows, and spending time listening to patients don’t cure disease, but they do provide comfort. No one wants to deny patients that kind of comfort. If the patient asks us about acupuncture or another objectively ineffective treatment, we can answer honestly with the negative scientific evidence while also acknowledging that some patients think they derive subjective comfort from it and that there is little chance of harm.
Conclusions
    • Placebos don’t really do much.
    • Placebos have no place in clinical practice. They are unethical.
    • Accepting false beliefs can harm and even kill. Using ineffective treatments can delay or interfere with effective conventional treatment.
    • Science-based medicine is the only reliable basis of establishing the safety and effectiveness of a treatment and developing rules for clinical practice, but we can sometimes justify bending those rules in the interests of patient comfort. Rigorous science can be tempered by common sense, benevolence, and empathy.
    • We can offer the patient comfort measures that will not change the course of the illness, as long as they are clearly identified as comfort measures with no false promises of therapeutic benefit.
    • We shouldn’t offer placebos, but we can sometimes condone them.
    • There is a fine line to walk between doing what patients think they want and what is in their best interests in the long run.
—————————-
Additions to the list in Part 1
31. X is officially approved by…so it must work.They cite some organization or authority. It may be Medicare, insurance companies, state licensing boards for acupuncture/chiropractic/naturopathy, the WHO, the courts, specific hospitals/clinics, individual doctors or experts. Also: the NCCAM is studying it, so there must be something to it.
This is similar to #7, the argument from popularity. It is a logical fallacy called the argument from authority. These organizations are not authorities when it comes to scientific truth; often they are not even experts in science. They are influenced by factors like politics, expediency, customer demand, economics, lobbyists, legal maneuvering, etc. No matter how many authorities approve of a treatment, it must still be properly tested to determine safety and efficacy.
32. I can’t afford conventional medicine; CAM costs less.If it costs less but doesn’t work, that’s false economy.
33. Studies show it doesn’t work, but what if it only works for me and a small minority of people like me?That’s possible, but not very probable. If it worked for a significant minority of people, that would have shown up in the data, would have affected the statistics, and would have changed the outcome of the study. If the minority was too small to affect the study outcome, what’s the likelihood that you would be one of the special few that it actually worked for? The odds are against it, and there is no rational way to choose the one that might work for you out of all the various treatments that have been tested and shown to be ineffective.
34. My doctor said nothing was wrong with me, but my CAM provider did a test conventional medicine doesn’t do, and found a condition that needed to be treated.How do you know it’s not one of the many bogus tests and bogus diagnoses that abound in the world of CAM?
35. Conventional medicine doesn’t have an effective treatment for my disease.CAM doesn’t either. They may tell you they do, but they will only offer false hope and waste your time and money. Maybe it’s time to accept that there is no effective treatment and concentrate on finding ways to cope and improve your quality of life.
36. Why do you concentrate so obsessively on CAM instead of attacking the abuses of conventional medicine and Big Pharma?This is sometimes phrased as “why don’t you put your own house in order before you criticize others?” We do frequently write about Big Pharma and about conventional medical practices. We don’t emphasize that, because conventional medicine is constantly criticizing and policing itself. Current practices are continually being re-evaluated and discarded if they are found ineffective. We concentrate on CAM because there is no such tradition of self-criticism in CAM, because they never reject any treatment when the evidence shows it doesn’t work, and because hardly anyone is writing critically about CAM.
This criticism amounts to saying “Why don’t you blog about what I consider to be important?” And the easy answer is: it’s not your blog, it’s ours. We get to decide.

Answering Our Critics – Again!
Critics of Science-Based Medicine keep making the same old tired arguments, despite the fact that their arguments have been repeatedly demolished. Here is a list of recurrent memes, with counterarguments.
Harriet Hall on February 13, 2018
Tweet
Instead of a new post this week I decided to recycle and revise what I wrote about Answering Our Critics a few years ago, here and here. I thought it was time to visit this issue again, because our critics didn’t get the message. They are still flooding the comments section with the same old tired arguments we have debunked over and over.
Some people don’t like what we have to say on Science-Based Medicine. Some attack specific points while others attack our whole approach. Every mention of complementary and alternative medicine (CAM) elicits protests in the Comments section from “true believer” users and practitioners of CAM. Every mention of a treatment that has been disproven or has not been properly tested elicits testimonials from people who claim to have experienced miraculous benefits from that treatment.
Our critics keep bringing up the same old memes, and I thought it might be useful to list those criticisms and answer them all in one place. I apologize for the length of this post; there were a lot of points to cover.
     1.  Big Pharma is paying you to promote their products and discredit CAM.No, it isn’t. We are not Pharma shills. We are not paid anything for writing this blog. We do not get money from pharmaceutical companies. We do not accept gifts from drug companies. We do not get kickbacks for prescribing certain drugs. We have no incentive to favor drugs over other treatments. Incidentally, critics who prefer natural remedies to pharmaceuticals should note that many of those dietary supplements are sold by subsidiaries of Big Pharma.
     2.  You’re biased.Yes, we are, and that’s a good thing. We are biased in favor of science and reason. We are biased against claims that have been tested and disproven and that are incompatible with the rest of scientific knowledge. We are biased against health care providers telling patients things that are not true, presenting opinions as if they were facts. We are biased against using placebos because we consider it unethical. We are not biased against any CAM treatment just because it is CAM; we contend that there is only one medicine, that treatments have either been proven to work or they haven’t, and that all claims should be held to the same standard and tested by the same scientific methods.
     3.  You’re afraid of the competition.Not at all. The SBM bloggers aren’t affected by “competition,” since we are either salaried or retired and don’t make more money by seeing more patients. Most doctors are overworked and already have all the patients they can handle. CAM has only a very small share of the healthcare market. It’s not that we are afraid CAM will take patients away from us, it’s that we are concerned for our patients’ welfare and don’t like to see them lied to, given ineffective treatments, persuaded to reject effective treatments, and sometimes harmed.
     4.  Science isn’t everything: there are other ways of knowing.When it comes to knowing whether a treatment is effective, there is only one reliable way of knowing: controlled testing using the scientific method. Intuition, tradition, revelation, “stoned thinking” à la Andrew Weil, dreams, extrapolation, speculation, and personal anecdotal experience can all lead people to strong beliefs, but we can’t trust those beliefs to reflect reality. Only the scientific method can give us reliable knowledge. No matter how convincing they sound, claims must be tested before we can assume they are true.
     5.  It worked for me.Maybe, maybe not. You can only know that you improved after the treatment; you can’t know for sure that you improved because of the treatment. That could be a post hoc ergo propter hoc logical fallacy. You may not be able to imagine any other possible explanation, but that doesn’t mean there isn’t one. Barry Beyerstein has explained some of the many ways people come to believe that a bogus therapy works. Also see the Quackwatch article on spontaneous remission and placebos.
     6.  Try it yourself.Trying it for yourself is not a reliable way to find out if a treatment works. If the symptoms resolve, you have no way of knowing whether they resolved due to the treatment or whether they would have gone away anyway without treatment, or whether some other factor caused the improvement. That’s why science uses control groups. And what if your symptoms don’t resolve? That doesn’t rule out the possibility that the treatment works for 99% of patients and you just happened to be one of the other 1%. If you try a remedy and get better, it’s reasonable on a practical basis to try it the next time you have the same symptoms, but it’s not acceptable to cite your experience as proof that “it works.”
     7.  Huge numbers of people use X, and they couldn’t all be wrong.Yes, they could. Popularity is no indication of truth. Over the centuries, how many people have believed in astrology? For centuries, everyone in Europe believed bloodletting was effective in treating diseases by balancing the humors. Only when it was properly tested did doctors discover they’d been killing patients instead of helping them.
     8.  It’s been used for centuries.This is the argument from antiquity and is a logical fallacy. Dr. Novella explains why. It could be ancient wisdom, but it could just as well be ancient error carried over from a prescientific era.
     9.  It’s natural, therefore it’s safe.Not necessarily. Many natural substances are poisons. Any natural remedy must be tested for efficacy and safety by the same standards we use to test “unnatural” remedies like pharmaceuticals. Here is Dr. Novella explaining the naturalistic fallacy. Herbs are drugs too, and anything that has an effect can have a side effect.
     10.  [Study] is proof that X is correlated with Y.Correlation does not prove causation. The rise in the number of diagnoses of autism correlates almost perfectly with the rise in the sales of organic food, but that doesn’t mean organic food causes autism. Apparent correlations can be due to chance, error, poor data collection, and many other things. There may not really be a true correlation; and even if there is, that doesn’t tell us whether X caused Y or Y caused X or whether X and Y might both be caused by Z.
     11.  There are hundreds of studies that show X works.In all too many cases, the studies supporters cite are animal or test tube studies, opinion pieces, speculations, and irrelevant studies. Even if there are human studies, there may be other, better quality human studies showing that it doesn’t work. Studies can be found to support almost any claim. Most published studies are wrong, for a variety of reasons that are constantly discussed on this blog. You can’t just look at positive studies: you have to look at the entire body of published evidence. That’s where systematic reviews come in. And they may not reflect reality: they may omit negative studies that we don’t know about because they were never published, due to the file drawer effect and publication bias. And remember Carl Sagan: “Extraordinary claims require extraordinary evidence.” It would take a great deal of evidence indeed to overthrow all the established science that tells us homeopathy can’t possibly work as advertised.
     12.  You are just robotically supporting the official party line of mainstream medicine.There is no official “party line,” but there is often a scientific consensus of experts. We don’t just blindly endorse the consensus; first we look at the evidence for ourselves. Most of the time we come to the same conclusions as the majority of mainstream doctors, because we are looking at the same evidence. When a body of experts evaluates all the published studies and makes evidence-based recommendations, we usually agree with them. Sometimes we disagree with their interpretation of the evidence (especially if they fail to consider prior probability or are including Tooth Fairy science) or with the way they have stated their conclusions. We understand that evidence-based guidelines are only guidelines, and judgment should be used in applying them to individual patients. We don’t agree just because we consider the experts authorities. There is a difference between the appeal to authority (“He’s a professor at Harvard, so we should believe everything he says!”) and accepting the consensus of experts who know more about the field than we do (if 10 top mechanics all agree that your carburetor needs replacing, it is only reasonable to assume that it probably needs replacing). All too often, CAM advocates are the ones who are parroting unreliable “authorities” who don’t know what they’re talking about.
     13.  Doctors only treat symptoms, not the underlying cause of disease.This silly CAM mantra is a vile, false accusation. Doctors treat the underlying cause whenever possible. If a patient has pneumonia, we don’t just treat the fever, pain, and cough; we figure out what microbe is responsible and provide the appropriate antibiotic. If a broken bone is painful, we don’t just treat the pain, we immobilize the fracture so it can heal. If a patient is in agony from pain in the right lower quadrant of the abdomen, we don’t just treat the pain, we try to figure out if the underlying cause is appendicitis, and if it is, we operate.
     13.  The people who accuse us of not treating “the underlying cause” are often the ones who think all disease is due to a single bogus underlying cause (subluxations, disturbances of qi, poor diet, etc.). They also tend to use a single treatment (when all you have is a hammer, everything looks like a nail). I once googled for “the one true cause of all disease” and found 63 of them. Science-based medicine recognizes 9 whole categories of causes, with the mnemonic VINDICATE.
     14.  Science-based medicine can’t explain why some people get a disease and others don’t.Neither can CAM. But doctors do have some pretty good ideas why it happens: exposure to people with infectious diseases, number of organisms that get into the body, genetic factors, toxins, immune deficiency, chance, and so on. CAM claims to fully understand why it happens, attributing it to some single cause that impairs optimum health (like a subluxation, or a disturbance in qi, or improper diet). But they have not been able to show they understand the answer to that question any better than conventional medicine does, or that their understanding leads to better patient outcomes.
     15.  Conventional medicine kills patients.I wrote about the “Death by Medicine” meme here. Critics gleefully cite statistics for drug reactions, medical errors, and iatrogenic deaths; but it is irrational to look at those numbers in isolation. Harms must be weighed against benefits. Medicine clearly saves lives. Many of those who develop treatment complications would have died even sooner without treatment. All effective treatments have side effects. We look at the risk/benefit ratios and reject treatments where the risk is greater than the potential benefit. The risk/benefit ratio of CAM should be compared to that of conventional medicine; if there is no proven benefit, no degree of risk can be justified. And mainstream medicine is constantly working to reduce the number of preventable errors; there is no similar effort in the world of CAM.
     16.  Your minds are closed.We are open to any new treatment, no matter how implausible, if only it can be shown to be safe and effective in well-designed controlled trials. Before we can ask how it works, we must ask if it works. If homeopathy had shown the same spectacular degree of success as penicillin, everyone would be using it. When Helicobacter was proposed as the cause of ulcers, it only took a few years for the evidence to accumulate and for antibiotics to become the treatment of choice. When a treatment like acupuncture has been studied for decades and even for centuries and its effectiveness is still uncertain, it is only reasonable to stop studying it and spend our research money elsewhere. We don’t need to keep an open mind about perpetual motion or a flat earth, and we don’t need to keep an open mind about homeopathy.
     17.  You are too prejudiced against CAM to look objectively at our evidence for it. No amount of evidence would change your minds.We change our minds all the time based on changing evidence. We look at the best evidence for a CAM treatment before we reject it. We would accept CAM if it could present the same quality and quantity of evidence that it takes to reach a scientific consensus about any other medical treatment. What would it take for CAM advocates to change their minds? Most of them hold their beliefs so firmly that they reject any evidence to the contrary. One practitioner told me he would keep using his pet method even if it was definitely proven not to work, because “his patients liked it.”
     18.  Science keeps changing its mind.Yes, and that’s a good thing. Scientific conclusions are always provisional. We follow the evidence wherever it leads, and we often have to change course as new evidence becomes available. CAM refuses to change its mind even in the face of clear evidence. Scientific medicine stops using treatments if they are proven not to work. CAM never rejects any treatment, and hardly ever tests one of its treatments against another to see which is superior.
     19.  Doctors are only out to make money.I think most doctors go into medicine not because they want to get rich but because they want to help people. Medical education is long, grueling, and expensive. Most doctors incur substantial debts for their education and need years to repay them. The nice houses and cars don’t come until long after graduation. Aren’t CAM providers out to make money? It’s much easier to get rich by pushing questionable treatments or spreading questionable information (cf Dr. Oz, Andrew Weil, Burzynski, Mercola, Natural News’ Mike Adams, Daniel Amen, Kevin Trudeau, and all the companies that sell dietary supplements and miracle weight loss aids). Boiron sold over half a billion Euros worth of homeopathic remedies in 2012.
     20.  Alternative treatments are individualized and can’t be subjected to the same tests as pharmaceuticals.Anything can and should be tested by scientific methods. For instance, homeopaths could prescribe individually in whatever way they chose, then the remedies they prescribe could be randomized with placebo controls and dispensed by someone else with double-blinding. Or the objective outcomes of treatment by conventional vs. CAM providers could be compared.
     21.  Doctors don’t do prevention.They most certainly do! Who do you think invented vaccinations and preventive screening tests? Don’t you know about the US Preventive Services Task Force? MDs routinely talk to patients about weight control, diet, seatbelts and other safety topics, alcohol, drugs, domestic violence, exercise, etc. Studies on these topics are constantly appearing in the major medical journals. And there’s no evidence that the preventive efforts of CAM providers result in any better health outcomes than those of MDs. In fact, patients of alternative practitioners have lower rates of immunization and other proven preventive measures than patients of MDs. And they are more likely to be diagnosed with a vaccine-preventable disease.
     22.  Doctors don’t know anything about nutrition.They understand the science of nutrition and advise their patients based on the available scientific evidence. They learn how vitamins, minerals, and other nutrients are utilized biochemically throughout the body. Doctors prescribe diets and refer patients to dietitians for the specific details about how to implement those diets. CAM providers claim to know more about nutrition, but they often give pseudoscientific or unfounded diet advice.
     23.  CAM is better because it’s holistic.CAM appropriated the holistic principle from mainstream medicine. Doctors are taught holistic principles in medical school. We are taught that the secret of the care of the patient is caring for the patient, not just treating the disease. Part of the standard medical history is a “social history.” Good clinicians consider the patient’s family, lifestyle, job, stresses, education, diet, socioeconomic status, beliefs, and everything about the individual that might have an impact on medical care.
     24.  We don’t need studies; we have plenty of testimonials.10 anecdotes are no better than one; 100 are no better than 10. Anecdotal evidence is unreliable, no matter how many anecdotes you have accumulated. This lesson has had to be relearned over and over again throughout the history of medicine. Just think of how many testimonials there were for bloodletting in the Middle Ages. Anecdotes are useful, but only as a guide to what to investigate with scientific studies.
     25.  Why won’t you believe us?We do. We believe you believe what you are telling us. We believe you had the experience you related. But that doesn’t mean your interpretation of your experience is true.
     26.  If you think X doesn’t work, why don’t you do a study to prove it?It’s not that we think X doesn’t work, it’s that there is no evidence to make us think it does work. It is not up to us to prove that it doesn’t work. It is up to the person making the claim to provide the supporting evidence. If I told you that putting a pea in your gas tank would give you better mileage, you should ask me to prove it. You shouldn’t feel obligated either to put a pea in your tank or to do a study to prove it didn’t work.
     27.  Natural remedies don’t get tested because they can’t be patented and there’s no profit in it.Nonsense. About half of our prescription drugs were developed from plants. The plant itself can’t be patented, but the drug company can isolate the active ingredient and patent that, or even improve on it with a synthetic version that is more effective and has fewer side effects. They can patent a unique method of converting a plant into a pill. There’s plenty of money to be made in herbal medicines, dietary supplements, and even plain old vitamins: they generate billions of dollars’ worth of profits every year.
     28.  The medical establishment would drum out any doctor who tried to publish studies going against the party line, showing that X worked or that condition Y was real.Quite the contrary. Peer review would critique the study. If it was a good study, others would investigate. A doctor who discovered a new disease or treatment would be honored. The treatment of ulcers with antibiotics is a case in point: Drs. Marshall and Warren won a Nobel Prize for their discovery. Montagnier was awarded a Nobel Prize for discovering the virus that causes AIDS only two years after the first reports of “gay-related immune deficiency syndrome.” Real diseases and effective new treatments are quickly recognized by the medical community.
     29.  You can’t know about it if you haven’t experienced it.You don’t have to have been bitten by a snake to know how to treat snakebite. Male obstetricians are proof that you can deliver babies without having been pregnant yourself. We can know that antibiotics work for pneumonia without having had pneumonia ourselves. You don’t have to have used Perkins tractors to know that they don’t work. In fact, personal experience is a handicap: it tends to interfere with the ability to objectively assess the evidence.
     30.  I can’t afford conventional medicine; CAM costs less.If it costs less but doesn’t work, that’s false economy.
     31.  Studies show it doesn’t work, but what if it only works for me and a small minority of people like me?That’s possible, but not very probable. If it worked for a significant minority of people, that would have shown up in the data, would have affected the statistics, and would have changed the outcome of the study. If the minority was too small to affect the study outcome, what’s the likelihood that you would be one of the special few that it actually worked for? The odds are against it, and there is no rational way to choose the one that might work for you out of all the various treatments that have been tested and shown to be ineffective.
     32.  My doctor said nothing was wrong with me, but my CAM provider did a test conventional medicine doesn’t do, and found a condition that needed to be treated.How do you know it’s not one of the many bogus tests and bogus diagnoses that abound in the world of CAM?
     33.  Conventional medicine doesn’t have an effective treatment for my disease.CAM doesn’t either. They may tell you they do, but they will only offer false hope and waste your time and money. Maybe it’s time to accept that there is no effective treatment and concentrate on finding ways to cope and improve your quality of life.
     34.  Why do you concentrate so obsessively on CAM instead of attacking the abuses of conventional medicine and Big Pharma?This is sometimes phrased as “why don’t you put your own house in order before you criticize others?” We do frequently write about Big Pharma and about conventional medical practices. We don’t emphasize that, because conventional medicine is constantly criticizing and policing itself. Current practices are continually being re-evaluated and discarded if they are found ineffective. We concentrate on CAM because there is no such tradition of self-criticism in CAM, because they never reject any treatment when the evidence shows it doesn’t work, and because hardly anyone is writing critically about CAM.This criticism amounts to saying, “Why don’t you blog about what I consider to be important?” And the easy answer is: it’s not your blog, it’s ours. We get to decide.
     35.  X is officially approved by…so it must work.They cite some organization or authority. It may be Medicare, insurance companies, state licensing boards for acupuncture/chiropractic/naturopathy, the WHO, the courts, specific hospitals/clinics, individual doctors or experts. Also: the NCCIH is studying it, so there must be something to it.This is a logical fallacy called the argument from authority. These organizations are not authorities when it comes to scientific truth; often they are not even experts in science. They are influenced by factors like politics, expediency, customer demand, economics, lobbyists, legal maneuvering, etc. No matter how many authorities approve of a treatment, it must still be properly tested to determine safety and efficacy.
     36.  If CAM makes people feel better, why deny them that? Even if it’s just a placebo, isn’t that a good thing?I would argue that we don’t just want our patients to think they are better, we want them to actually be better. A study that illustrates that principle has been discussed on this blog before, here and here. Asthma patients in the placebo groups thoughtthey were better; but their lung function was unchanged, which could have led to fatal consequences. Placebos don’t really do much, and it is unethical for doctors to prescribe them. Deception tends to undermine the doctor/patient relationship. There is no place for the use of placebos outside of clinical research studies.
     37.  What’s the harm?CAM advocates tend to pass through these stages:
             (i)  They accept testimonial evidence and see no need for scientific testing.
             (ii)  They recognize that the scientific community will not accept their claims without scientific testing.
             (iii)  They do some studies.
             (iv)  When flaws in the studies are pointed out, they try to do better studies.
             (v)  A systematic analysis of all studies fails to support their claims.
             (vi)  They are forced to admit that there is no convincing scientific evidence that their treatment works better than placebo.
             (vii)  They argue that placebos are “effective” and that it is acceptable to lie to patients.
             (viii)  They argue that placebos are harmless, and even if the treatment is only a placebo, it should be used for its placebo benefit.
     37.  Belief in the efficacy of CAM can lead patients to reject or delay effective treatment. Patients have died. There is a whole website, What’s the Harm, devoted to the harm that false beliefs can do. So far, it has compiled reports of 368,379 people killed, 306,096 injured and over $2,815,931,000 in economic damages. Please go there and look at some of the accounts of patients who have died or suffered serious consequences because of false medical beliefs about everything from acupuncture to vitamin megadoses.
Conclusion: a handy guide to debunking
Our critics will inevitably continue to make the same tired old arguments, and when they do, you can provide them with a link to this list or cut and paste from it to answer their criticisms. I hope you find this list useful. Counterarguments won’t change the minds of true believers, but they may persuade other readers who haven’t irrevocably made up their minds.

   Blog Discussion with an SBM Critic
Steven Novella
Link
Over the last couple of days I have been engaged at NeuroLogica in a discussion with a fellow blogger, Marya Zilberberg who blogs at Healthcare, etc. Since the topic of discussion is science-based medicine I thought it appropriate to reproduce my two posts here, which contain links to her posts.
A Post-Modernist Response to Science-Based Medicine
I receive frequent commentary on my public writing, which is great. The feature that most distinguishes blogs is that they are conversations. So I am glad to see that science-based medicine (a term I coined) is getting targeted for criticism in other blogs. One blogger, Marya Zilberberg at Healthcare, etc., has written a series of posts responding to what she thinks is our position at Science-based medicine. What she has done, however, is make many of the logical fallacies typically committed in defense of unscientific medical modalities and framed them as one giant straw man.
She is partly responding to this article of mine on SBM (What’s the harm) in which I make the point that medicine is a risk vs benefit game. Ethical responsible medical practice involves interventions where there is at least the probability of doing more benefit than harm with proper informed consent, so the patient knows what those chances are. Using scientifically dubious treatments, where there is little or no chance of benefit, especially when they are overhyped, is therefore unethical. And further, the “harm” side of the equation needs to include all forms of harm, not just direct physical harm.
Zilberberg’s response is the typical tu quoque logical fallacy — well, science-based medicine is not all it’s cracked up to be either, so there. She writes:
Now, let’s get on to “proof” in science-based medicine. As you well know, while we do have evidence for efficacy and safety of some modalities, many are grandfathered without any science. Even those that are shown to have acceptable efficacy and safety profiles as mandated by the FDA, are arguably (and many do argue) not all that. There is an important concept in clinical science of heterogeneous response to treatment, HTE, which I have addressed extensively on my blog. I did not make it up, it is very real, and it is this phenomenon that makes it difficult to predict how an individual will respond to a particular intervention. This confounds much of what we think is God’s own word on what is supposed to work in allopathic medicine.
This is also the fallacy of the perfect solution — since science-based medicine is not perfect, there is no legitimate basis for criticism of any modality. This is also premised on the false dichotomy of “allopathic medicine” (a derogatory term only used, in my experience, by defenders of dubious medicine) vs “alternative” medicine (which I will refer to as CAM for short). I and others at SBM have been clear that we eschew this false dichotomy. There is only medicine with varying degrees of plausibility and evidence — there is a continuum, and we advocate always using the best that is available. We also think there should be a minimum standard, a fuzzy line of plausibility and evidence below which treatments should only be given with proper informed consent as part of an approved clinical trial. And a further line below which even research is unethical because there is no plausible potential for benefit.
These principles are, in fact, already part of ethical medicine. We did not invent these concepts. It is, rather, the proponents of CAM who wish to do away with this ethical standard — to create a false dichotomy in order to establish a double standard. We are not trying to create a new standard, just to do away with the double standard of CAM.
She refers to the heterogeneous response to treatment, again as if realization of this basic fact is not already part of science-based medicine. I, in fact, explain this to patients all the time. Our knowledge of treatments is based upon statistics, but we can never know ahead of time how an individual patient will respond. What’s the alternative? Until we get better at predicting individual response (which really will just be another application of statistics), this is the best we can do. That is why you monitor the individual response to any treatment, and act accordingly. This is basic medical student stuff, but Zilberberg acts as if this is a big revelation for science-based medicine.
We at SBM advocate for the highest scientific standards of medicine, and apply that across the board — including with pharmaceutical companies, surgeons, and anything else that is labeled as “mainstream.” Again — we do not make false categories and distinctions. It is all medicine.
The reference to “God’s own word” is an obvious allusion to the bad-old-days of paternalistic medicine (dead and buried for decades now), or the TV caricature of a doctor with a God complex. This is a typical ploy — portray any attempt at defending a scientific standard in medicine as paternalistic arrogance. In fact, Zilberberg dedicates an entire blog post to this fallacy. She writes:
First of all, it is my belief that all interventions should be approached with equanimity, if not equipoise. Although I am quite dubious that either healing crystals or Reiki can produce actual results, I do not want to confuse the absence of any evidence to this effect with the evidence of absence of the effect. Although I am not that interested in allocating resources to studying these fields, it would be paternalistic of me to bar their further investigation. So the society can decide what it wants to do with them, and in the meantime every individual can make her/his own choice whether to spend their money on them.
This is clearly where we differ. I do think, from reading her writing, that Zilberberg means well and is sincere in her positions (unlike some I criticize who I feel are just trying to sell something). But notice the logical contortions in her position — she wants “equipoise” with regard to all interventions, and would not dare dictate how research money is spent. I should point out that there is a range of opinions on SBM when it comes to regulation — so we are not a united front on this score. We range from libertarians who think that we should educate the public and professionals, but are against laws that would restrict access to unscientific modalities. Essentially, people have the right to make stupid decisions. Others believe that there needs to be a minimum safety net against fraud and quackery, and in fact the public wants there to be one and believes there already is one. I don’t want to get bogged down in this debate on this blog entry — I am just pointing out that Zilberberg’s premise is overly simplistic and paints with too broad a brush.
But the real point here is that she is taking an almost post-modernist position that we need to approach all claims in medicine with “equipoise.” She says that society can decide how research money is spent, even if she would not personally research an implausible topic. Depending upon how you slice it, this is not necessarily far off from my position. If people want to raise money to research an implausible question they should go right ahead. I never proposed banning implausible research. My position, rather, is that we should not waste limited public/government research resources on highly implausible modalities.
I would also add, however, that once you start doing research on humans there is a host of ethics that also come into play. In human research it is the accepted ethical standard that subjects should at least have a chance of benefiting from the treatments being studied, or at least there should be a greater chance of benefit than harm. I don’t see how this ethical standard can be met with homeopathy, for example, where there is essentially zero chance of benefit. At some point you pass a line of infinitessimal plausibility where the ethics become problematic.
Zilberberg then makes the “absence of evidence vs evidence of absence” mistake — really an oversimplification of this concept to the point of being wrong. While it is true that the absence of evidence if not necessarily evidence of absence — it can be, depending upon how thoroughly you have looked. If I search my house for a specific item and don’t find it, that is pretty good evidence that it’s not there. It is not “proof” of absence, but it is evidence. With many of the modalities that Zilberberg admits she is personally dubious about there is evidence of absence of an effect. This evidence comes in two forms — all of the science that tells us the modalities are highly implausible, and often there is clinical evidence of lack of an effect. To pretend otherwise is dishonest — it is hiding from the facts out of political correctness.
Further, our patients do not want equipoise from us. They want our informed opinion. When patients ask me if they should take a homeopathic remedy I don’t give them a wishy washy answer. I give them my informed opinion, and they are grateful to have it. In the comments to her blog a commenter speculates about my bedside manner, assuming, essentially, that I must be a paternalistic ass. This is the typical cardboard caricature I encounter, and it has no relationship to reality. It is possible to give patients useful information without being judgmental. To give them informed consent (how do you do this, by the way, without giving them information?) but understand that they will make up their own minds. Patients are in charge of their own health care, and our job as clinicians, more than ever, is to give them the information and perspective they need to make good decisions. This does not demand “equipoise”, but evidence and perspective. In my opinion equipoise in the face of ridiculously implausible claims and evidence of lack of efficacy is a disservice to patients and a violation of trust.
Ironically, Zilberberg concludes:
Bottom line, we need to appreciate that none of the science is all that straightforward. Let us not dumb down the arguments and create false dichotomies. If we do, no one wins.
Does she actually read science-based medicine? I am left to wonder — since we regularly argue for the complexity of the science of medicine. I want people to understand how complex the relationship is, so they are not shocked every time conflicting studies come out. Medical science is a messy business, and it is challenging often to infer what the best approach is. I want the profession and the public to have a much more nuanced understanding of medical science, and for the media to do a better job of representing it.
This is especially true since we do not have a paternalistic system. Patients are partners in their own health care, and therefore it helps me do my job when they understand the science that underpins medicine.
Zilberberg’s position is anti-science, although perhaps not deliberately so. It is anti-science in a post-modernist sense. She points out all the limitations of science, as if that means we cannot come to any meaningful decision, and therefore must treat all claims as equal. But all claims are not equal. Even the best are imperfect, but we can still apply science and evidence to make informed decisions about the probability of risk vs benefit. And there are some claims that are so against science and evidence (like homeopathy) that any stance other than rejection is a violation, in my opinion, of medical ethics and the trust that society places in medical professionals.
In Zilberberg’s world, however, any such judgments are the equivalent of pronouncing that these treatments over here in pile A are deemed “scientific” (as if by the word of God) and are accepted. And these over here in pile B are deemed “nonsense” and are to be ridiculed. But the false dichotomy is in her mind, not in science-based medicine. We are the ones railing against the false dichotomy — that of CAM which seeks to create a double standard. All we advocate is one consistent standard of science and evidence when evaluating all medical claims, and the rational application of science to the practice of medicine.
One final note — I would much prefer to have a conversation with the critics of science-based medicine that does not constantly involve defending SBM and myself from false accusations of arrogance and paternalism. I think it says a lot about their intellectual position when that is constantly the best they have.
Dr. Zilberberg Responds
Dr. Zilberberg responded to my original post and significantly modified her tone, to her credit. (She was simultaneously responding to Orac’s analysis of her posts as well.) Here is my analysis of that post.
The Tone Thing
I will address her main points below, but first my final thoughts on the “tone” thing. While she admits fault in setting the “confrontational tone,” I don’t think she quite gets what Orac and I were objecting to. I actually don’t mind a confrontational approach — as long as it is substantive (that’s the way science works — if you have a point to make, bring it on). We were objecting to her mischaracterizing our position and making ad hominem attacks in place of substantive criticism — essentially using the “arrogant” gambit with which we are all too familiar. Her readers obviously picked up on this, and piled one, accusing us of being bullies and thanking her for slapping us down. We objected to her logical fallacies, not her tone.
Interestingly Zilberberg’s initial response was dismissive, and she reiterated the charge of paternalism and arrogance, writing: “If the shoe fits?” At least now she seems to realize that if we are going to have a productive discussion, focusing on ad hominem attacks will be counterproductive.
Incidentally, having written about medicine for years I have definitely seen a strong pattern. When I criticize the logic and factual premises of another person’s argument I am frequently accused of being mean by people who then attack me personally. It seems many people do not understand the difference between a strong but substantive criticism and a personal attack. Zilberberg was falling into this category, but has significantly (if incompletely) backed off from that with her latest post.
One more minor point — “allopathic” is derogatory and does not apply to modern medicine (it was coined by Samuel Hahneman to refer to the poisons that passed for medicine in his time, and was definitely meant to be a criticism). I would suggest she drop this term rather than defend it.
Evidence in Medicine
Zilberberg then launches into a meaty discussion of what her position actually is. She observes that perhaps we are not that far off in our positions, which I think is true. There is a meaningful difference in spin — the final conclusions drawn from the analysis, but her analysis of the role of evidence in medicine is reasonable. But again, to clarify, Orac and I were not objecting to the point that evidence in medicine is messy and complex. We were objecting to the accusation that we do not understand this, and that we are promoting an overly simplistic and cheerleading approach to science in medicine. This left me with the impression that Zilberberg has not read deeply into the Science-Based Medicine website, or at least has failed to grasp what it is we are actually saying.
If she had she would have seen post after post in which SBM authors were pointing out all of the complexities and deficiencies of evidence in medicine that she and others might also point to. That is core to the point of SBM — evidence is complex. She might, in fact, have read my series of posts on evidence in medicine. We do spend a great deal of time pointing this out in the context of so-called CAM, because CAM proponents are the ones who most profoundly take a simplistic approach to the evidence. They engage in black-and-white thinking, display intolerance of ambiguity, and frequently advocate for the reliance on very problematic low-grade evidence to support their claims. But we also consistently apply the same standards to surgery and the pharmaceutical industry, and anything “mainstream.’
Zilberberg reviews the relative roles of experimental evidence vs observational evidence. Her analysis is reasonable, but I think she overstates the utility of observational data a bit (and she admits to a fondness for this type of data). The bottom line is that each type of evidence (basic science, observational, and experimental — and even anecdotal) has its own strengths and weaknesses, and the best result comes from analyzing all kinds of scientific evidence and looking for a consensus of evidence. That is, in fact, OUR criticism of evidence-based medicine -over reliance on randomized controlled clinical trials and undervaluing other forms of scientific evidence. That is why we advocate for “science”-based medicine, and not just “evidence”-based medicine.
Each type of evidence, in fact, is abused. We criticize the inappropriate extrapolation from basic science to clinical claims, assuming causation from observational correlation, failure to realize the limits of clinical trials, and the use of pragmatic studies as if they were evidence for efficacy.
Zilberberg also clarifies her position by saying that she feels there is good scientific evidence for some of medicine, but it seems she differs from my position in how evidence-based modern medicine actually is.
We can argue endlessly about this question — how much of modern medicine is based upon solid evidence — each pointing to limited examples and essentially giving our bias. But there are some facts we can point to. Zilberberg writes:
While it is true that the oft-cited 5-20% number representing the proportion of medical treatments having solid evidence behind them is very likely outdated, the kind of evidence we are talking about is a different matter.
The “5-20% number” is not outdated — it’s a myth. Actually, I had previously heard 15% as the low end, but I guess that number keeps dropping. I wrote previously about this myth here. The 15% number was based upon an extremely small survey of primary care practices in the north of England — in 1961. That’s almost 50 years ago. The number was never very relevant, and now it’s a joke.
More recent surveys of medical practice come to very different numbers. Bob Imrie reviewed the published evidence:
Thus, published results show an average of 37.02% of interventions are supported by RCT (median = 38%). They show an average of 76% of interventions are supported by some form of compelling evidence (median = 78%).
Of course, where you draw the line for “supported by compelling evidence” will determine what the percentage figure is. But the bottom line is that the 15% figure is basically an urban legend, and “5%” is nothing short of propaganda. More reasonable estimates range much higher.
And — the point of EBM and SBM is that we can and should do better. We also need to do better in adhering to EBM guidelines where they exist, and in utilizing continuing medical education and other mechanisms of quality control to improve adherence to the evidence where it does exist.
The difference in spin is not subtle. We can look at the evidence and say: modern medicine has a culture of science, endeavors to be scientific, and basically the system works but the process is complex and messy and there are multiple ways in which we can do better. Meanwhile someone else can look at the same data and conclude: modern medicine is broken, it is based upon arrogance, authority, and greed, and we can just throw up our hands and conclude that any treatment is as likely to be of value as any other, no matter how silly it may seem scientifically.
My position is essentially the former. Zilberberg came off originally as being close to the latter (and judging by the comments, many readers took her position to be supportive of the latter), but now has clarified that she is somewhere in the middle.
CAM
Zilberberg also clarifies her position on CAM. She had previously written that she advocates a position of “equipoise” towards clinical claims. Even though she might not use certain modalities herself, she sees no basis to condemn the use of them by others. I characterized this position as political correctness gone wild — to the point of practical post-modernism. Now she writes:
My belief is that all modalities that may impact what happens to public’s health need to be evaluated for safety, not question. I think we both agree, since there is really no reason to think that something like homeopathy has anything that can help, by the same token we do not believe that it have anything that can hurt. Same with healing crystals, reiki and prayer. So, if a person wants to engage in these activities, and they are perfectly safe physically, be my guest. Other modalities, such as chiropractic, acupuncture, herbalism and the like, definitely need to be evaluated more stringently, as there is reason to think that they may cause harm.
This is a common position to take. Val Jones at SBM coined the term “shruggie” to refer to this position — in essence, if there is no direct harm, then who cares what people do. First, as I discussed very recently on SBM, there are many types of harm from unscientific medical modalities other than direct physical harm. So I do not find this position tenable for that reason alone.
Further, context is everything. There are actually a variety of positions that authors at SBM take when it comes to regulating medical practice. We all generally believe that medical professionals should not engage in nor promote unscientific methods. In fact, we should oppose their adoption and promotion, we should oppose their inclusion in universities and mainstream hospitals, and spending public funds on researching extremely implausible or already disproven modalities. That seems to be a point of difference between myself and Zilberberg.
I personally do not oppose individuals doing whatever they want when it comes to their own health. If you want to chew on tree bark (a vivid example given to me by someone else), go right ahead. What I object to is someone selling the tree bark and claiming that it cures cancer based upon nothing but legend and anecdote, and scaring their customers away from proven therapies in order to make the sale. I object to distortions of logic and science in order to confuse the public so as to better market worthless or harmful products. And I object to medical professionals looking the other way out of misguided political correctness, or simply a naivete as to the significant harm that is done.
SBM has a huge consumer protection mission, and it puzzles frustrates me that this mission is so often and so thoroughly misrepresented. This misrepresentation is deliberate — part of the “health freedom” movement — and seeks to portray all health care consumer protection activity as arrogant elitism and protectionism. This is identical to the intelligent design movement’s representation of all attempts at quality control in education as arrogant elitism.
What I don’t understand is Zilberberg’s apparent position that, while she knows homeopathy is utterly worthless, a physician should refrain from telling her patients exactly that.
Vaccine Skepticism
Zilberberg goes on to argue that she is not anti-vaccine, as she has been accused (not by me). I have no reason not to accept her word on this, and it is good that she has clarified her position.
But I do think she is displaying a lack of appreciation for the nature of the anti-vaccine movement. As an example, if one publicly expresses doubt about an aspect of currently accepted Darwinian evolution it would be nice if they understand the many ways in which the scientific discourse is exploited by creationists, so that they don’t accidentally give succor to an anti-scientific movement.
Likewise, any public discussion about vaccines, while it should be candid and completely honest, should ideally be done with an adequate familiarity with the anti-vaccine movement’s propaganda so that one’s words and positions are not easily exploited. In fact, while expressing skepticism about a particular vaccine or vaccine program, I would recommend specifically clarifying one’s position to distance themselves from the extremists. Otherwise you are inviting misinterpretation.
Conclusion
The take home message from this exchange is that, in my opinion, accusations of using harsh tone or of arrogance are an ad hominem distraction from the real issue — what is the optimal relationship between the practice of medicine and the underlying science of medicine.
Zilberberg engaged fully in this distraction, but is now slowly backing away (but not enough, in my opinion). I think this was largely due to the fact that she has been taken in by the very active and sophisticated propaganda campaigns of CAM proponents. She seems to have bought into their rhetoric, and did not read carefully enough into our writing at SBM to see through it.
We are approaching 1000 blog posts at SBM. I don’t expect critics to read every post, but a tiny bit of scholarly due diligence would be nice, before essentially buying into the lies and distortions of our critics.
We at SBM write frequently about the complexity and limitations of the science of medicine. That is our mantra — a nuanced and sophisticated approach to evidence is needed. But at the end of the day, some treatments are better than others. We can accept and reject practices based upon plausibility and evidence, even while there is a vast gray zone in the middle where we just don’t know yet.
It is misleading and ironic in the extreme to criticize promoters of SBM for taking a simplistic approach to evidence. That is the opposite of the truth. Meanwhile, promoters of all sorts of so-called CAM do take a simplistic and highly distorted approach to evidence, display an intolerance of uncertainty, systematically misrepresent the evidence to their clients and the public, think in stark black-and-white terms, engage in bait-and-switch deceptions, distort the positions of their critics, rely upon low grade evidence and logical fallacies for their claims, and then hide behind political correctness, post-modernism, distractions about “health care freedom”, special pleading (science can’t test my claims), and accusations of arrogance and paternalism.
All of this behavior is carefully documented in the pages of Science-Based Medicine. Would-be critics of SBM should try reading some of them before launching into misguided criticism of what is ultimately a straw man of our actual positions.
I take Zilberberg at her word that she is interested in genuine discussion, and she has at least moved in that direction. I recommend she step back, read some more of SBM and see what we actually have to say about science and medicine.

Responding to SBM Critics
A response to a critic of SBM, and setting the record straight on our actual positions regarding evidence and the practice of medicine.
Steven Novella on April 19, 2017
Link

I started this blog not only to create a useful resource but to have a conversation. As our tagline suggests, we want to explore (not dictate) the relationship between science and the practice of medicine. As it turns out, that relationship is very complicated. That conversation takes place in the comments, but also by responding to other outlets who are, in turn, responding to us.
No doubt, we are promoting a specific approach to medicine that we call science-based. Briefly, we feel that medical interventions that are safe and effective are inherently superior to interventions that are unsafe or ineffective (that should be a hopefully non-controversial premise). We also feel, as the name implies, that the best way to determine which interventions are safe and effective is by looking critically at all the available science. This includes examining the process and institutions of medical science themselves, how information is reported and consumed, the role of various types of bias, different types of evidence, and thinking carefully about where we should place the threshold of evidence when informing clinical practice and health consumer decisions.
Further we feel strongly that there should be one, thoughtful, science-based standard that applies uniformly to all of health care.
We spend a considerable amount of time pushing back against those who explicitly disagree with this approach, and who are promoting other approaches to medicine and health. Many of these non-SBM approaches have been marketed under the umbrella brand of “alternative,” “complementary,” or “integrative” medicine.
We know and expect that doing what we do will attract critics, and we are happy to have them. I tend to respond directly to critics when either I think they have a point, or I think they are misrepresenting our position and I want to use a response as an opportunity to set the record straight. Sometimes the best way to explain a concept is to respond directly to someone who has a misconception about that concept.
Responding to SelfHacked
In that spirit, I want to address some of the points made in a blog post on the website SelfHACKED. The post is from a few years ago (although we just noticed it), but is still attracting comments. Also, I checked and there do not appear to be any updated posts on the site about SBM, so this still stands as their latest opinion. Some of the criticisms may have therefore been addressed by subsequent posts on SBM, so to be fair I cannot hold them to account for articles we wrote after their post.
Here is how the editors of SelfHacked describe their philosophy:
At Selfhacked, we plan on dissecting every facet of human biology and looking at potential remedies or ways to reverse our biological weaknesses – a first principle’s [sic] approach to health and medicine. This will all be in a format that will be accessible to everyone.People need to take control of their own health and not rely on anyone. And the best weapon for that is solid information, combined with a spirit of self-experimentation.
With all of that in mind, let’s take a look at some of their specific criticisms. They do not get off to a good start:
With regard to health, true skepticism means to be doubtful in a belief that some treatment works, but at the same time also be doubtful in forming opinions that something doesn’t work.This means you shouldn’t form beliefs one way or the other. Your attitude should be one of not knowing if you want to be truly skeptical.If you read SBM posts, you will see they are skeptical of ‘alternative’ remedies, but they believe that these remedies don’t work.If you ask them about any remedy that isn’t proven by science, they will tell you the chance is close to zero that it works. They believe the people who feel an effect from them are just experiencing the placebo effect.
This is both a straw man and a false premise. Being skeptical does not mean you do not form opinions. That definition is closer to the philosophical skeptics who argue against all knowledge, but that is not what modern skepticism promotes. We are scientific skeptics. That means you use science and reason to inform tentative conclusions about what is likely to be true.
Further, medicine is applied science. This means you have to make decisions. That is the whole point of SBM, to discuss how science informs medical decisions. You can’t make decisions if your position is endlessly one of not knowing. Rather, you should ask, what is the best decision we can make at the current time considering all the scientific evidence in the context of risk vs benefit and personal preference.
But of course people don’t like to be told that their preferred treatment probably doesn’t work, especially if they have tried it and feel that it does. Some go as far as claiming that we can never know that something does not work, which is convenient if you want to promote dubious treatments.
Now for the straw man – it is demonstrably not our position that anything not proven by science has a close to zero chance of working. We have never taken such a position explicitly, nor can you infer that position from what we write. Our stated position is that plausibility and clinical evidence are two distinct considerations. A treatment may be plausible but untested, or highly implausible. We certainly treat such claims differently.
Any fair read of SBM articles will show that we are very careful not to confuse unproven with disproven, and lack of evidence for efficacy with evidence for lack of efficacy. Just yesterday Harriet wrote:
Vital Stem is a dietary supplement mixture that supposedly reverses the changes of normal aging by increasing the body’s production of stem cells. We can’t know if it works, because it hasn’t been tested.
In my experience this criticism comes from a careless read of what we actually write, through a filter biased by a specific outlook. There is also often a specific agenda to support the position that SBM has an unreasonably high standard for evidence, and therefore we reject treatments that might actually work. More on this point below.
Next they make an odd point about plausibility. They argue that, sure, some things like homeopathy are incredibly implausible, but herbs and supplements are not.
All 300 or so supplements/interventions that I’ve tried had a noticeable effect (except grounding). The question isn’t if they have an effect, but what that effect is and if it benefits you more than it harms you.I know supplements have an effect because I keep upping the dosage until I’m certain I feel an effect.
This is another straw man, implying we argue that herbs as a category are implausible. We have explicitly stated that herbs and supplements, because they are actual chemicals with potential pharmacological effects, are toward the plausible end of the spectrum when it comes to unproven therapies. Our criticisms, rather, are that herbs are drugs – they are dirty, unpurified, unregulated, and understudied drugs. Most fail due to poor bioavailability.
I agree with the authors in that the important question is – what effects do specific herbs have and can they be used in a way that the benefits outweigh the harm? I strongly disagree with their method for answering this question, self-experimentation (i.e. anecdotal evidence). I would not, by the way, just keeping upping the dose of a substance until you feel something. You might not feel something until after your kidneys shut down. This is, in fact, reckless and horrible advice.
The authors at SelfHacked take the explicit position that the best way to know if something works is to take it. They brush off criticisms about placebo effects and the difficulty of determining efficacy from subjective assessments and an N of 1. In typical fashion they partly push back against SBM’s criticism of this position by making another straw man out of our position. We do not say that the patient’s experience is unimportant. Of course, if a drug does not work for a specific patient we would not advocate using it. You have to individualize treatment. Anecdotal evidence also can be useful in forming hypotheses to be tested.
But you cannot determine efficacy from anecdotes. This has been clearly established over the last century of scientific medicine. Anecdotes were used to support radioactive tonics, and worthless radio machines, and countless other inert treatments that we now know with a high degree of confidence do not work. Anecdotes are much more likely to mislead you than inform you. This is clear, and you have to be in abject denial or ignorance of history to disagree.
More straw men coming (you see a pattern here):
Here’s how to know SBMs opinion before you read any article. If you find that even one of their thousands of articles don’t fit this paradigm, please post it.
    • If it’s natural, it’s not effective.
    • If a study has shown a natural product effective, it’s flawed.
    • If the study isn’t flawed, it’s clinically insignificant.
    • If the FDA claims something is safe, SBM will never disagree.
    • If a drug is FDA approved, it’s effective. SBM will never question its efficacy or ‘clinical significance’.
This is a clear ad hominem fallacy, or at least attempt at poisoning the well. The implication is that our opinions are ideological and biased. This dismissive approach replaces a detailed analysis of our specific opinions, which we always back up with a detailed analysis of the basic and clinical science. If you can’t argue against the details of someone’s position, it is easier to brush it aside with a broad dismissal.
None of what is stated above is an actual position of ours, not philosophically or in practice. If there is a kernel of truth to some of the characterizations, it is not for the reason implied. While SBM advocates for one standard across all of medicine, we do spend a disproportionate amount of our time dealing with the more fringe end of the spectrum. This is a deliberate editorial choice. This reflects our niche, not our philosophy, and we have stated this numerous times.
In addition to our clinical specialties, we all share a common specialty in critically analyzing pseudoscience. We don’t need to spend our time doing systematic reviews of mainstream treatments, because other experts are fully capable (and probably better equipped) to do so. Our efforts would be redundant there. This does not mean we don’t think it’s important. That is just not our niche.
Where EBM and mainstream medicine fall down is in putting pseudoscience into a thoughtful context, because they are not familiar with it. We are.
One common form of pseudoscience in today’s culture is to promote a treatment as “natural.” The explicit claim is that something which is natural is better and more likely to be safe and effective than something which is synthetic. First, this is a false dichotomy, and there is no operational definition of what is “natural.” Second, this is simply not true. Being derived from a natural source says nothing about safety or efficacy. Many natural substances are terrible poisons. Therefore, by necessity, we spend a lot of time pushing back against the appeal to nature fallacy.
This does not mean we argue a “natural” substance cannot be effective. That is simply absurd. We never said it or implied it. In fact we have used the argument that there are many substances derived from natural sources that are part of mainstream medicine, including many of our modern pharmaceuticals. We have directly endorsed pharmacognosy, which is the scientific study of natural substances.
The next two points are similarly silly. First, as we have explicitly stated many times, most studies are flawed, and one way in which they are flawed is that effect sizes are small and not clinically significant. SBM is about promoting a reasonably high standard of evidence, and it is very common to use studies that are either cherry picked or fatally flawed to promote treatments that are not adequately supported by evidence.
Regarding the FDA, they have a very high standard for safety. It is not perfect, but it is reasonable. This does not mean no side effects or risk free, and drugs do slip through the cracks (as the Vioxx scandal shows). It is not our position that they are always correct, just that their safety standards are sufficiently high. If the authors think we have taken the wrong position on a specific drug, then they should point it out. You may notice, by the way, that the article is extremely light on specific examples. That should serve as a big clue. In fact, they state that sometimes that are criticizing our apparent “attitude” rather than our stated position. This is a bit convenient, and allows them to just offer their subjective opinion without backing it up with specific examples.
Regarding efficacy, again, for drugs, the FDA has a pretty high standard. But, some things still slip through the cracks and we will criticize this. The Cephaly device for migraines, for example, I stated was prematurely approved based on unconvincing evidence. We have had frank discussions of the efficacy of the flu vaccine, and of antidepressants for mild to moderate depression. This claim is simply at odds with the evidence.
Next comes their biggest whopper:
The type of evidence that SBM advocates is a random threshold of evidence. There’s no reason it can’t be lower.
This claim shows that they have not read SBM to any serious degree. We spend a lot of time exploring the exact question of where the threshold of evidence should be an why. To say this threshold is “random” is to ignore literally hundreds of articles we have written on the subject. In fact I wrote an article called “evidence thresholds” in 2013, before their criticism.
There are multiple independent lines of evidence that we have specifically examined and used to support our position regarding evidence thresholds. We have discussed evidence from Ioannidis about the predictive value of positive studies (in fact, most are wrong), evidence from reproducibility attempts, from the Simmons et al. article on p-hacking, from research on reversals, on the predictive value of promising basic science, and examining the history of individual treatments. We have examined the relative value of p-values vs Bayesian analysis. We have deliberately built what we feel is a very strong case, based on a long and growing list of published studies, for where the threshold of evidence should be. To dismiss this as “random” is beyond laughable.
I see such criticism as part of a broader agenda – to specifically lower the standards of evidence in medicine to admit whatever it is that someone wants to promote that is not backed by adequate evidence.
There is a lot more nonsense, but here is one final point:
I really don’t like the term science-based medicine, because it paints an inaccurate picture.Much of the scientific knowledge we have is based on animal studies, yet we call it science.However, SBM is not referring to this type of science with regard to medicine. They are referring to large, replicated, double-blind, placebo-controlled trials, published in the top 3 journals.
This is actually the opposite of our position. One of our specific criticisms of EBM is that they rely too heavily on placebo-controlled clinical trials, and ignore other types of evidence, such as basic science. We have specifically stated this numerous times. We call ourselves science-based, because we think all science should be considered, not just clinical evidence.
Where the authors are profoundly confused is that we also discuss the context of each type of evidence. We don’t dismiss evidence, we just put it into its proper context.
Preclinical studies are useful for determine plausibility and to some degree safety. They are not adequate for determining efficacy. A tiny percentage, less than 1%, of promising treatments from animal or other basic science studies turn out to be useful in humans. That is a fact.
So, basic science is good for plausibility and for preliminary safety. Preliminary clinical trials are unreliable but inform later more definitive clinical trials. Only replicable rigorous clinical trials have any real predictive value when it comes to efficacy – but these studies need to be put into the context of prior plausibility based on basic and other clinical science. Pragmatic studies are useful in informing real-world use of treatments, but cannot be used to determine efficacy.
Again – these are all noncontroversial positions generally accepted by actual scientists who know what they are talking about. We are not saying anything new or radical, just pointing this out and making sure it is not forgotten when considering individual questions in medicine.
It is actually our critics who are taking a new and radical position, one in opposition to the published evidence. They are the ones trying to change (specifically reduce) the usual standards of evidence that have evolved over the last century. If anything we are just trying to tweak those standards in light of recent research which informs how predictive and reliable current medical research is.
Conclusion: The need for charity
I invite the authors of the SelfHacked criticism to respond to this response, and also to update their article after actually reading some relevant SBM posts on the topics of their criticism.
I will also point out that there is a general lesson to learned from the hack job offered by the SelfHacked authors. If you are going to criticize someone else’s position, you should make a sincere effort to understand what their position actually is, and to be fair to their position (the principle of charity). Also back up your claims with specific examples. If you can’t, there may be a reason for that.

   Answering a criticism of science-based medicine
David Gorski
Link
Attacks on science-based medicine (SBM) come in many forms. There are the loony forms that we see daily from the anti-vaccine movement, quackery promoters like Mike Adams and Joe Mercola, those who engage in “quackademic medicine,” and postmodernists who view science as “just another narrative,” as valid as any other or even view science- and evidence-based medicine as “microfascism.” Sometimes, these complaints come from self-proclaimed champions of evidence-based medicine (EBM) who, their self-characterization otherwise, show signs of having a bit of a soft spot for the ol’ woo. Then sometimes there are thoughtful, serious criticisms of some of the assumptions that underlie SBM.
The criticism I am about to address tries to be one of these but ultimately fails because it attacks a straw man version of SBM.
True, the criticism of SBM I’m about to address does come from someone named Steve Simon, who vocally supports EBM but doesn’t like the the criticism of EBM implicit in the very creation of the concept of SBM. Simon has even written a very good deconstruction of postmodern attacks on evidence-based medicine (EBM) himself, as well as quite a few other good discussions of medicine and statistics. Unfortunately, in his criticism, Simon appears to have completely missed the point about the difference between SBM and EBM. As a result, his criticisms of SBM wind up being mostly the application of a flamethrower to a Burning Man-sized straw man representing what he thinks SBM to be. It makes for a fun fireworks show but is ultimately misdirected, a lot of heat but little light. For a bit of background, Simon’s post first piqued my curiosity because of its title, Is there something better than Evidence Based Medicine out there? The other reason that it caught my attention was the extreme naiveté revealed in the arguments used. In fact, Simon’s naiveté reminds me very much of my very own naiveté about three years ago.
Here’s the point where I tell you a secret about the very creation of this blog. Shortly after Steve Novella invited me to join, the founding members of SBM engaged in several e-mail frank and free-wheeling exchanges about what the blog should be like, what topics we wanted to cover, and what our philosophy should be. One of these exchanges was about the very nature of SBM and how it is distinguished from EBM, the latter of which I viewed as the best way to practice medicine. During that exchange, I made arguments that, in retrospect, were eerily similar to the ones by Simon that I’m about to address right now. Oh, how epic these arguments were! In retrospect, I can but shake my head at my own extreme naiveté, which I now see mirrored in Simon’s criticism of SBM. Yes, I was converted, so to speak (if you’ll forgive the religious terminology), which is why I see in Simon’s article a lot of my former self, at least in terms of how I used to view evidence in medicine.
The main gist of Simon’s complaint comes right at the beginning of his article:
Someone asked me about a claim made on an interesting blog, Science Based Medicine. The blog claims that Science Based Medicine (SBM), that tries to draw a distinction between that practice and Evidence Based Medicine (EBM). SBM is better because “EBM, in a nutshell, ignores prior probability (unless there is no other available evidence and falls for the p-value fallacy; SBM does not.” Here’s what I wrote.
No. The gist of the science based medicine blog appears to be that we should not encourage research into medical therapies that have no plausible scientific mechanism. That’s quite a different message, in my opinion, that the message promoted by the p-value fallacy article by Goodman.
First off, Simon’s complaint makes me wonder if he actually read Dr. Atwood’s entire post. To show you what I mean, I present here the whole quote from Dr. Atwood in context:
EBM, in a nutshell, ignores prior probability† (unless there is no other available evidence) and falls for the “p-value fallacy”; SBM does not. Please don’t bicker about this if you haven’t read the links above and some of their own references, particularly the EBM Levels of Evidence scheme and two articles by Steven Goodman (here and here). Also, note that it is not necessary to agree with Ioannidis that “most published research findings are false” to agree with his assertion, quoted above, about what determines the probability that a research finding is true.
Simon, unfortunately, decides to bicker. In doing so, he builds a massive straw man. I’m going to jump ahead to the passage the most reveals Simon’s extreme naiveté:
No thoughtful practitioner of EBM, to my knowledge, has suggested that EBM ignore scientific mechanisms.
Talk about a “no true Scotsman” fallacy!
You know, about three years ago I can recall writing almost exactly the same thing in the aforementioned epic e-mail exchange arguing the very nature of EBM versus SBM. The problem, of course, is not that EBM completely ignores scientific mechanisms. That’s every bit as much of a straw man characterization of SBM as the characterization that Simon skewered of EBM being only about randomized clinical trials (RCTs). The problem with EBM is, rather, that it ranks basic science principles as being on either very lowest rung or the second lowest rung on the various hierarchies of evidence that EBM promulgates as the way to evaluate the reliability of scientific evidence to be used in deciding which therapies work. The most well-known of these is the that published by the Centre for Evidence-Based Medicine, but there are others. Eddie Lang, for instance, places basic research second from the bottom, just above anecdotal clinical experience of the sort favored by Dr. Jay Gordon (see Figure 2). Duke University doesn’t even really mention basic science; rather it appears to lump it together at the very bottom of the evidence pyramid under “background information.” When I first started to appreciate the difference between EBM and SBM, I basically had to be dragged, kicking and screaming, by Steve and Kimball, to look at these charts and realize that, yes, in the formal hierarchies of evidence used by the major centers for EBM, basic science and plausible scientific mechanisms do rank at or near the bottom. I didn’t want to accept that it was true. I really didn’t. I didn’t want to believe that SBM is not synonymous with EBM, which would be as it should be in an ideal world. Simon apparently doesn’t either:
Everybody seems to criticize EBM for an exclusive reliance on randomized clinical trials (RCTs). The blog uses the term “methodolatry” in this context. A group of nurses who advocate a post-modern philosophical approach to medical care also criticized EBM and used an even stronger term, micro-fascism, to describe the tendency of EBM to rely exclusively on RCTs.
But I have not seen any serious evidence of EBM relying exclusively on RCTs. That’s certainly not what David Sackett was proposing in the 1996 BMJ editorial “Evidence based medicine: what it is and what it isn’t”. Trish Greenhalgh elaborates on quite clearly in her book “How to Read a Paper: The Basics of Evidence Based Medicine” that EBM is much more than relying on the best clinical trial. There is, perhaps, too great a tendency for EBM proponents to rely on checklists, but that is an understandable and forgivable excess.
I must to admit to considerable puzzlement here. EBM lists randomized clinical trials (RCTs) and meta-analyses or systematic reviews of RCTs as being the highest form of evidence, yet Simon says he sees no serious evidence of EBM relying exclusively on RCTs. I suppose that’s true in a trivial sort of way, given that there are conditions and questions for which there are few or no good RCTs. When that is the case, one has no option but to rely on “lower” forms of evidence. However, the impetus behind EBM is to use RCTs wherever possible in order to decide which therapies are best. If that weren’t true, why elevate RCTs to the very top of the evidence hierarchy? Simon is basically misstating the the complaint anyway. We do not criticize EBM for an “exclusive” reliance on RCTs but rather for an overreliance on RCTs devoid of scientific context.
Simon then decides to try to turn the charge of “methodolatry,” or as revere once famously called it, the profane worship of the randomized clinical trial as the only valid method of investigation, against us.This misinterpretation of what SBM is leads Simon, after having accused SBM of leveling straw man attacks against EBM, to building up that aforementioned Burning Man-sized straw man himself, which he then begins to light on fire with gusto:
I would argue further that it is a form of methodolatry to insist on a plausible scientific mechanism as a pre-requisite for ANY research for a medical intervention. It should be a strong consideration, but we need to remember that many medical discoveries preceded the identification of a plausible scientific mechanism.
While this is mostly true, one might point out that, once the mechanisms behind such discoveries were identified, all of them had a degree of plausibility in that they did not require the overthrow of huge swaths of well-settled science in order to be accepted as valid. Let’s take the example of homeopathy. I use homeopathy a lot because it is, quite literally, water and because its proposed mechanism of action goes against huge swaths of science that has been well-characterized for centuries. I’m not just talking one scientific discipline, either. For homeopathy to be true, much of what we currently understand about physics, chemistry, and biology would have to be, as I am wont to say, not just wrong, but spectacularly wrong. That is more than just lacking prior plausibility. It’s about as close to being impossible as one can imagine in science. Now, I suppose there is a possibility that scientists could be spectacularly wrong about so much settled science at once. If they are, however, it would take compelling evidence on the order of the mass of evidence that supports the impossibility of homeopathy to make that possibility worth taking seriously. Extraordinary claims require extraordinary evidence. RCTs showing barely statistically significant effects do not constitute extraordinary evidence, given that chance alone will guarantee that some RCTs will be positive even in the absence of an effect and the biases and deficiencies even in RCTs. Kimball explains this concept quite well:
When this sort of evidence [the abundant basic science evidence demonstrating homeopathy to be incredibly implausible] is weighed against the equivocal clinical trial literature, it is abundantly clear that homeopathic “remedies” have no specific, biological effects. Yet EBM relegates such evidence to “Level 5”: the lowest in the scheme. How persuasive is the evidence that EBM dismisses? The “infinitesimals” claim alone is the equivalent of a proposal for a perpetual motion machine. The same medical academics who call for more studies of homeopathy would be embarrassed, one hopes, to be found insisting upon “studies” of perpetual motion machines. Basic chemistry is still a prerequisite for medical school, as far as I’m aware.
Yes, Simon is indeed tearing down a straw man. As Kimball himself would no doubt agree, even the most hardcore SBM aficianado does not insist on a plausible scientific mechanism as a “pre-requisite” for “ANY” research, as Simon claims. Rather, what we insist on is that the range of potential mechanisms proposed do not require breaking the laws of physics or that there be highly compelling evidence that the therapy under study actually has some sort of effect sufficient to make us doubt our understanding of the biology involved.
Simon then appeals to there being some sort of “societal value” to test interventions that are widely used in society even when those interventions have no plausible mechanism. I might agree with him, except for two considerations. First, no amount of studies will convince, for example, homeopaths that homeopathy doesn’t work. Witness Dana Ullman if you don’t believe me. Second, research funds are scarce and likely to become even more so over the next few years. From a societal perspective, it’s very hard to justify allocating scarce research dollars to the study of incredibly implausible therapies like homeopathy, reiki, or therapeutic touch. (After all, reiki is nothing more than faith healing based on Eastern mystic religious beliefs rather than Christianity.) Given that, for the foreseeable future, research funding will be a zero sum game, it would be incredibly irresponsible to allocate funds to studies of magic and fairy dust like homeopathy, knowing that those are funds that won’t be going to treatment modalities that might actually work.
When it all comes down to it, I think that Simon is, as I was, in denial. When confronted with the whole concept of SBM compared to EBM, I denied what I didn’t want to believe. To me, it seemed so utterly obvious that the scientific plausibility of the hypothesis under study has to be taken into account in evaluating the evidence. I just couldn’t imagine that any system of evaluating evidence could be otherwise; it made no sense to me. So I imposed this common-sense view on EBM, and I rather suspect that many other advocates of EBM like Simon labor under the same delusion I did. The problem is, though, that critics of EBM are basically correct on this score. Still, realizing it or admitting it did not come easy. For me to accept that EBM had a blind spot when it came to basic science, it took having my face rubbed in unethical and scientifically dubious trials like that of the Gonzalez therapy for pancreatic cancer or chelation therapy for cardiovascular disease. Let’s put it this way. To be willing to waste money studying something that is nothing but water and has as its “scientific basis” a hypothesis that is the equivalent of claiming that a perpetual motion machine can be constructed tells me that basic science basically means close to nothing. Ditto wasting money on studying a therapy whose major component is coffee enemas used to treat a deadly cancer. Simon cheekily suggests at the end of his post that “maybe we should distinguish between EBM and PIEBM (poorly Implemented Evidence Based Medicine). The problem is, trials of therapies like the Gonzalez regimen, homeopathy, and reiki are a feature of, not a bug in EBM. In fact, I challenge Simon to provide a rationale under EBM as it is currently constituted to justify not having to do a clinical of these therapies. There is none.
I realize that others have said it before here (and probably said it better than I), but we at SBM are not hostile to EBM at all. Rather, we view EBM as incomplete, a subset of SBM. It’s also too easily corrupted to provide an air of scientific legitimacy to fairy dust like homeopathy and reiki. These problems, we argue, can be ameliorated by expanding EBM into SBM. Personally, I suspect that the originators of EBM, as I do (and, I suspect, Simon does), never thought of the possibility of EBM being applied to hypotheses as awe-inspiringly implausible as those of CAM. It simply never occurred to them; they probably assumed that any hypothesis that reaches a clinical trial stage must have good preclinical (i.e., basic science) evidence to support its efficacy. But we know now that this isn’t the case. I can’t speak for everyone else here, but after agreeing with Kimball that EBM ought to be synonymous with SBM I also express the hope that one day there will be no distinction between SBM and EBM. Unfortunately, we aren’t there yet.

   Answering another criticism of science-based medicine
David Gorski
Link
In the three and a half years that the Science-Based Medicine blog has existed, we contributors have come in for our share of criticism. Sometimes, the criticism is relatively mild; often it’s based on a misunderstanding of what SBM is; but sometimes it’s quite nasty. I can’t speak for the rest of the SBM crew on this, but I’ve gotten used to it. It comes with the territory, and there’s little to do about it other than to skim each criticism as it comes in to see if the author makes any valid points and, if he doesn’t, to ignore it and move on. Indeed, there’s enough criticism being flung our way that I rarely respond directly anymore. Exceptions tend to be egregious examples, incidents that spark real problems, such as when Age of Autism blogger and anti-vaccine activist Jake Crosby tried to paint me as being hopelessly in the thrall of big pharma, which resulted in the anti-vaccine horde who read that blog to try to get me fired by sending complaints to the Board of Governors at my university and the dean of my medical school. Other examples tend to be what I call “teachable moments,” in which the mistakes made in the criticism provide fodder for making a point about SBM versus alternative medicine, “complementary and alternative medicine” (CAM), or “integrative medicine” (IM)—or whatever the nom du jour is.
File this next one under the “teachable moment” variety of criticism directed at SBM.
A pain researcher takes Steve and me to task
In order to stoke my massive ego (if you don’t believe I have a massive ego, just ask my critics), I have a Google Alerts set to my name. One thing I learn from these alerts is that there are a lot of people out there named David Gorski, few of them doing medicine and not all of them particularly reputable. The other thing I learn is that I’m actually mentioned on the web and in the blogosphere more times than I ever would have imagined possible a few years ago. More importantly, I’m quickly aware of criticisms directed my way. So it was that I found out about an article by someone named Stewart B. Leavitt, MA, PhD entitled A Burgeoning Klatch of Science Skeptics, which managed to attack both Steve Novella and myself in a manner that’s so off-base that it presents said “teachable moment,” much as an earlier criticism by Steve Simon provided such a moment.
You know right off the bat that Dr. Leavitt is going to go far off base when his opening paragraph contains the passage:
As it turns out, skepticism regarding modern science, including the field of pain medicine, is somewhat of an international movement. Its mantra might be “Stop the B.S.” (with the irreverent logo at right); yet, there is an apparent danger of skepticism becoming close-minded cynicism falling into the trap of contempt prior to investigation that might preclude truly objective analyses of research.
The community of skeptics, worldwide, serves an important function in modern society, as they always have. It was probably skeptics who asked, “What makes you think the Earth is flat — where’s your proof?” When it comes to healthcare, skeptics at their best evaluate medical treatments and products in a scientific light, promoting the highest standards and traditions of scientific inquiry; however, at their worst, skeptics may merely be naysayers, stirring up the pot of already-established evidence to draw attention to themselves.
This is the classic confusing of skepticism with nihilism or cynicism. In my experience, sometimes skepticism and cynicism are conflated intentionally by opponents of SBM in order to dismiss skepticism as nothing more than a cynical “Dr. No” approach in which claims are reflexively rejected without consideration, but sometimes (and perhaps more frequently) the two are conflated through ignorance because the difference between the two is not understood. I rather suspect that Dr. Leavitt does not understand the difference. (At least, I will give him the benefit of the doubt.) And this criticism can be true to a point: As skeptics we have to be very careful to avoid falling into the trap of cynicism, of rejecting claims without giving them a fair hearing. Indeed, most of us have said as much ourselves on this very blog on multiple occasions, particularly how we ourselves have to try to avoid falling into the trap of motivated reasoning, which is wielded to great effect by proponents of pseudoscience to protect themselves from having to change their minds about cherished views. On the other hand, how many times do clinical trials of pseudoscience like, for example, homeopathy have to fail and how often do we have to point out that the principles by which homeopaths claim homeopathy works violate well-established laws of physics and chemistry before we’re allowed the shorthand of provisionally rejecting homeopathy until its proponents produce data compelling enough to make us question the laws of physics as currently understood?
I won’t take on Leavitt’s criticism of Steve Novella (much). After all, Steve is more than capable of taking care of himself, having been in the skeptic biz at a higher level several years longer than I have. I do note, however, that, although Leavitt links to both Steve’s blog and SBM, he does not directly link to the posts that he cites, making it difficult to see what is being said in context. For example, Leavitt cites Novella as saying:
I have become thoroughly convinced of the axiom that there is no claim so absurd that it cannot attract flocks of true believers. The default mode of human psychology is to think with our emotions, then deftly rationalize our decisions. As a result there do not appear to be any practical limits to human gullibility.
An excellent quote, succinctly summing up the situation. It comes from a post in which Steve deconstructed an example of pure health supplement quackery. In the context of Steve’s post, it makes perfect sense. Yet Leavitt dismisses this as an “emotive” appeal. Similarly, he is very unhappy when I say things that he views as similarly emotive, in particular, when I refer to the “integration” of alternative medicine modalities into medicine as “quackademic medicine” and refer to “integrative medicine” as “integrating quackery with medicine.” I’ll be the first to admit that I came up with these sorts of pithy phrases (aside from “quackademic medicine,” whose origin I cannot claim, as much as I would like to) to make a point in a memorable way that can be easily repeated. This is how one communicates complex topics in a sound bite world, and it worked at TAM in that my point about “integrating quackery with medicine” was Tweeted and re-Tweeted rather extensively.
Of course, it is rather amusing to note that Leavitt dismisses our appeals as “emotive,” while he himself denigrates skeptics using language at least as emotive as ours, describing us, in essence, as cynics who “posture” and preach to the choir in the proverbial echo chamber, such as meetings like TAM, which Leavitt describes as:
The most recent TAM brought together skeptics from around the world and a long list of guest speakers and panelists. Workshops included: Defending Evolution, Skeptical Activism , Advancing Skepticism Online, Investigating Monster Mysteries, Raising Skeptics, and others. And, among the sessions discrediting UFOs and paranormal phenomena were the usual attacks against “alternative therapies” in healthcare and pain management.
Note the dismissive tone about sessions including the “usual attacks” against “alternative therapies.”
More telling is Leavitt’s attempt to contrast the type of skepticism that he claims he promotes, which he characterizes as “healthy skepticism” and “educated skepticism,” with our skepticism. He begins with a nonsequitur:
Along with that, however, we have acknowledged that it is far easier to criticize science — asking tough questions, pointing out flaws or weaknesses — than it is to do good science. Therefore, we were somewhat dismayed by the writings and posturing of the self-proclaimed community of professional skeptics.
So what if it’s easier to criticize science and to point out flaws or weaknesses than it is to do good science? Even if true, this observation on Leavitt’s part serves no other purpose besides denigrating critics of CAM. As someone who does what I like to think is good science myself and has been funded by the NIH, the Department of Defense, and the Conquer Cancer Foundation of ASCO, I find Leavitt’s comment profoundly insulting. His argument boils down to, in essence, a claim that if you don’t do pain research science you aren’t qualified to criticize bad CAM research about pain treatments. From my perspective, though, good science is good science, and bad science is bad science. It doesn’t matter who is doing the criticism; what matters is whether the criticisms are valid. In fact, Leavitt’s use of language is as clever as that of the CAM proponents we routinely discuss. His skepticism is “healthy” and “educated.” Left unspoken is what our skepticism must be—presumably by contrast “unhealthy” and “uneducated”—while Leavitt also characterizes us as “posturing.”
Logical fallacies and false dichotomies on parade
After the warmup, Leavitt gets to the meat of his objections, or so it would appear:
For example, their diatribes against CAM and integrative therapies — which actually can be vital modalities for effective pain management — seem guided more by emotional arguments than a systematic study of all available evidence. Hence, when Novella writes (as noted above), “The default mode of human psychology is to think with our emotions, then deftly rationalize our decisions. As a result there do not appear to be any practical limits to human gullibility,” we wonder if he also is describing how a credulous community of skeptics approach their subjects of scorn.
And, when Gorski asserts that “integrative medicine is all too often in reality nothing more than ‘integrating’ pseudoscience with science, quackery with medicine,” we might assume he has solid evidence to justify such claims; but, if so, he is keeping it secret. And, his far-reaching denigration of NCCAM, the Bravewell Collaborative, and unnamed medical schools adds emotive impact to arguments that might have no basis in fact.
Leavitt amuses me in this passage. His first criticism of Steve is a deftly executed tu quoque argument, in which he seems to concede that the default mode of human psychology is to think with our emotions (which it is, by the way) but turns it around to accuse skeptics of the same thing. Here’s the difference, though. Skeptics know that the default mode of human thought is to think with our emotions, to leap to conclusions first and then to try to find evidence to justify our opinions. Skepticism, science, and critical thinking are all methods designed to try to minimize that very human tendency and to minimize the effects of the cognitive quirks we all share that mislead us, including confusing confirmation bias, confusing correlation with causation, regression to the mean, and placebo responses.
I’m further amused that Leavitt would think that I don’t provide solid evidence to justify my claims. Once again, he doesn’t directly link to the post from which that quote comes, which is entitled The ultimate in “integrative medicine,” continued, which described a cooperative agreement between Georgetown University School of Medicine and the Bastyr University, the latter of which is a school of naturopathy. Clearly, Leavitt doesn’t support the notion that much of naturopathy is quackery and pseudoscience. Even if that’s the case and he disagrees, Leavitt reveals pure laziness in his statement, given that I’ve written copiously, logorrheically even, about this issue, providing numerous examples. For example, here are a few:
     1.  The ultimate in “integrative medicine”: Integrating the unscientific into the medical school curriculum
     2.  The ultimate in “integrative medicine,” continued
     3.  A University of Michigan Medical School alumnus confronts anthroposophic medicine at his alma mater
     4.  An open letter to NIH Director Francis Collins regarding his appearance at the Society for Integrative Oncology
     5.  “Integrative” oncology: Trojan horse, quackademic medicine, or both?
     6.  The National Center for Complementary and Alternative Medicine (NCCAM): Your tax dollars hard at work
     7.  Cancer Treatment Centers of America and “naturopathic oncology”
     8.  NCCAM Director Dr. Josephine Briggs and the American Association of Naturopathic Physicians
     9.  Surprise, surprise! Dr. Andrew Weil doesn’t like evidence-based medicine
As I particularly glaring example selected from the posts above, I submit that the “integration” of anthroposophic medicine with real medicine at my alma mater is the integration of quackery and pseudoscience with medicine.
I’ve also provided multiple lines of evidence on my other blog, and my fellow SBM bloggers have written about this issue, in particular Kimball Atwood. Gratifyingly, we’re even starting to have some success, as the Cochrane Reviews editors have been showing signs of starting to “get” SBM.
Then, after stating that neither Steve nor I have taken the time to study the issues of “integrative medicine” in depth, Leavitt concedes:
Admittedly, the comments above from Gorski and Novella are mere snippets of their voluminous writings, described out of context. Readers can themselves visit the blogs of these skeptics (linked above) to decide the merits of their arguments. For example, we find using words like “pseudoscience” and “quackery,” as they do, to be rather slanderous and empty invectives.
In other words, Leavitt admits to quoting us out of context and then just links to our blogs in general, all the while not providing a single concrete example of our committing the offenses of which he accuses us. These tactics are disingenuous at best and intellectually dishonest at worst, particularly given that Leavitt never bothered to link directly to the posts from which he extracted our comments out of context. Basically, this gambit allows Leavitt to give the appearance of being fair while misrepresenting our arguments and trying to paint us as emotion-driven, biased, and “uneducated” about CAM. Think of it this way. It’s asking a lot—and I do mean a lot—for anyone to visit two blogs that have been in existence for several years, each with thousands of posts, and to try to evaluate the core of the voluminous arguments there. What will happen is that without guidance regarding where to start even the small minority who click on links in a blog post will be quickly overwhelmed by the volume of verbiage and tend simply to accept Leavitt’s characterization without too much investigation.
Leavitt concludes with a false dichotomy:
The lesson in all this is that our understanding of “healthy skepticism,” as advocated in these Pain-Topics UPDATES, may have an objective of “stopping the B.S.,” but it is not necessarily bent on “spoiling anyone’s fun.” And, it is not a close-minded approach that encourages contempt prior to adequate investigation.
Skepticism in pain research should drive a search for truth that recognizes and acknowledges the boundaries of uncertainty. Unfortunately, a great deal of research in the pain field seems driven by political agendas and hidden self-interests, so educated skeptics have much work ahead. We hope our readers are up to the task.
There are two problems here. (Actually, there are more than that, but I’ll concentrate on two.) First, as I said, it’s a false dichotomy. Either you accept Leavitt’s dismissive definition of skepticism, or you’re necessarily “spoiling someone’s fun.” And make no mistake, Leavitt is a pretty credulous fellow. For instance, he has made a holiday “wish list” for “holistic care” that includes a number of pseudoscientific alternative medicine modalities—is that an oxymoron?—including acupuncture, ayurveda, energy medicine (healing touch and reiki), and traditional Chinese medicine, among others. About homeopathy, Leavitt says, “Homeopathy is a therapeutic modality practiced worldwide, it is very popular among some patients, and it has withstood the test of time,” while expressing disappointment in a study of homeopathy for rheumatoid arthritis showed no therapeutic effect and trying to attribute the negative result of the study to its being underpowered. In yet another example, Leavitt critiques Edzard Ernst’s recent review of acupuncture systematic reviews, and while much of what he writes in this particular post is not unreasonable, he can’t help but drop bombs of credulity on the science, saying, for example, that “comparing acupuncture to a molecular entity, such as a drug for pain, may reflect a Western bias regarding how medical treatments are expected to work” and citing an acupuncture proponent who concludes that “it is probably no coincidence that many positive trials of [acupuncture and related techniques] have come from China where the techniques have been practiced for centuries; whereas, studies conducted in other countries often use divergent forms of acupuncture that also may be hindered by poor or improper technique, such as using only a limited number of sites or incorrect acupoints.”
Leavitt then concludes about acupuncture:
Considering the multitude of patients worldwide who have benefitted from acupuncture in one way or another, it still appears premature to broadly dismiss it as being of little or no value for pain relief.
This is yet another appeal to popularity devoid of science.
Finally, I can’t help but cite one last example in which Leavitt discusses “biofield therapies” (reiki, healing touch, etc.):
Although many traditionally-trained practitioners may remain skeptical, significant numbers of patients apparently seek biofield therapies, often without telling their healthcare providers, and the techniques have been used over millennia in various cultures to allegedly heal physical and mental disorders. In general, complementary and alternative therapies are used by 38% of adults and 12% of children in America, and it is a $34 billion per year business; so, these approaches cannot be easily ignored. The customary caveat — more research is necessary to arrive at definitive conclusions — would seem very appropriate regarding biofield-based therapies for pain. However, as the 16th Century Swiss physician Philipus Aureolus Paracelsus advised, “The art of healing comes from nature, not from the physician. Therefore the physician must start from nature, with an open mind.”
I submit that this more than approaches being so open-minded that one’s brains fall out. It embraces such unskeptical open-mindedness and gives it a big, sloppy kiss on the lips. Basically, what we have here is an appeal to popularity plus an argument from ignorance; i.e., that something is likely to be true simply because it hasn’t been proved false or, equivalently, that something is likely to be false because it hasn’t been proved true. Of course, for modalities like energy healing and homeopathy, it’s a fallacy to say that we don’t know. Basic science, such as physics, can tell us with a high degree of probability that homeopathy can’t work, for instance.
Dr. Leavitt’s straw man caricature of skepticism versus positive skepticism
Near the end of his post, Leavitt defines skepticism as “encouraging contempt prior to adequate investigation.” At the risk of sounding “contemptuous,” I am going to say that this is a fetid load of dingo’s kidneys. It’s a favorite straw man characterization of skepticism by the credulous, be they believers in the paranormal, alternative medicine aficionados, evolution denialists, or other promoters of pseudoscience. Personally, in response to such arguments, I like to cite the writings of Michael Shermer regarding positive skepticism:
This brings me to the larger issue of two forms of skepticism, negative and positive. Stephen Jay Gould began his foreword to my 1997 book, Why People Believe Weird Things, by noting: “Skepticism or debunking often receives the bad rap reserved for activities — like garbage disposal — that absolutely must be done for a safe and sane life, but seem either unglamorous or unworthy of overt celebration.”…
Positive skepticism, however, involves much more than the negative disposal of false claims. In fact, the word “skeptic” comes from the Greek skeptikos, for “thoughtful.” According to the Oxford English Dictionary, “skeptical” has also been used to mean “inquiring,” “reflective,” and, with variations in the ancient Greek, “watchman” or “mark to aim at.” What a positive meaning for what we do! We are thoughtful, inquiring, and reflective, and we are the watchmen who guard against bad ideas in order to discover good ideas, consumer advocates of critical thinking who, through the guidelines of science, establish a mark at which to aim. “Proper debunking is done in the interest of an alternate model of explanation, not as a nihilistic exercise,” Gould concludes. “The alternate model is rationality itself, tied to moral decency —the most powerful joint instrument for good that our planet has ever known.”
In other words, not only is “negative skepticism” not cynicism, but it’s actually a good thing. Indeed, skepticism is far more than just the debunking of claims. It’s the application of the best of human reason, including science, logic, and critical thinking to claims. When properly done, skepticism represents the highest intellectual aspirations of humankind. As Shermer points out, skepticism keeps the borders of science from moving too far into the realm of pseudoscience and non-science, and for every Copernicus, Newton, and Einstein, there are hundreds, if not thousands, of cranks and quacks whose ideas never pass scientific muster because they are nonsense. It might be some day that a CAM modality such as homeopathy or energy healing will pass scientific muster. Unlikely, but possible, and if and when that day comes I will examine the evidence and, if appropriate, change my mind to embrace what I formerly considered pseudoscience. That’s what skepticism really is. Moreover, science-based medicine is not an attempt to turn medicine into a pure science; rather it is the philosophy that science should inform and guide medicine.
To conclude, I realize that I perhaps cite this too often and that some might be offended by its language, which some might consider NSFW, but Tim Minchin put it best in his beat poem Storm:
Science adjusts its beliefs based on what’s observed
Faith is the denial of observation so that Belief can be preserved.
If you show me that, say, homeopathy works,
Then I will change my mind
I’ll spin on a…dime
I’ll be embarrassed as hell,
But I will run through the streets yelling
It’s a miracle! Take physics and bin it!
Water has memory!
And while its memory of a long lost drop of onion juice is Infinite
It somehow forgets all the poo it’s had in it!
You show me that it works and how it works
And when I’ve recovered from the shock
I will take a compass and carve “Fancy that!” on the side of my cock.
And I will, too. Well, except perhaps for the part about the compass and carving. We wouldn’t want to go too far, now, would we? On the other hand, one wonders how rapidly Dr. Leavitt will abandon beliefs that he possesses that are not supported by science. Apparently not very quickly. After all, he remains “open-minded” to energy healing, acupuncture, and other unscientific forms of medicine and shows no sign of changing.

Do clinical trials work? It depends on what you mean by “work”
David Gorski on July 15, 2013
Link
Introduction
(Skip to the next section if you want to miss the self-referential blather about TAM.)
As I write this, I’m winging my way home from TAM, crammed uncomfortably—very uncomfortably—in a window seat in steerage—I mean, coach). I had been thinking of just rerunning a post and having done with it, sleeping the flight away, to arrive tanned, rested, and ready to continue the battle against pseudoscience and quackery at home, but this seat is just too damned uncomfortable. So I might as well use the three and a half hours or so left on this flight to write something. If this post ends abruptly, it will be because I’ve run out of time and a flight attendant is telling me to shut down my computer in those cloyingly polite but simultaneously imperious voices that they all seem to have.
I had thought of simply recounting the adventures of the SBM crew who did make it out to TAM to give talks at workshops and the main stage and to be on panels, but that seems too easy. Even easier, I could simply post my slides online. But, no, how on earth can I reasonably expect Mark Crislip to post while he’s at TAM if I’m too frikkin’ lazy to follow suit? I’m supposed to lead by example, right, even if what comes out is nearly as riddled with spelling and grammar errors (not to mention the occasional incoherent sentence) as a Mark Crislip post? Example or not, lazy or not, I would be remiss if, before delving into the topic of today’s post, I didn’t praise my fellow SBM bloggers who were with me, namely Steve Novella, Harriet Hall, and Mark Crislip, for their excellent talks and insightful analysis. Ditto Bob Blaskiewicz, with whom I tag-teamed a talk on everybody’s favorite cancer “researcher” and doctor, Stanislaw Burzynski. It’ll be fun to see the reaction of Eric Merola and all the other Burzynski sycophants, toadies, and lackeys when Bob’s and my talks finally hit YouTube. Sadly, we’ll have to wait several weeks for that. (Hmmm. Maybe I will post those slides later this week.)
Finally, before I delve into the meat of the post, I do have to suggest to you all one last thing. Please go back and reread this post. Those of you who were at Penn Jillette’s Private Rock & Roll Bacon & Donut Party this year will understand the reason for the request. Those who weren’t (i.e., the vast majority of you) will, I hope, find it worth reading again. Let’s just say that Penn apparently actually read it, his profanity-laden protestations otherwise at the party notwithstanding, and was unhappy enough with me both to kick me out of his party, and have a song prepared about the incident to sing at the party. Let’s also just say that his choice of reaction wasn’t one of Penn’s finer moments, even by Penn’s usual standards of decorum. (Stay classy, Penn. Stay classy.)
Now that I’ve completed the obligatory 400-500 self-indulgent, introductory blather that no one cares about but is nonetheless a mandatory part of nearly every Gorski post because I can’t help myself and need an editor, let’s get to it. Just be thankful that it’s only 500 words this time.
The real post: Do clinical trials work?
One of the issues I discussed at our SBM workshop was something I’ve written before, namely the “methodolatry” that sometimes infests evidence-based medicine (EBM), “Methodolatry” has been defined as the profane worship of the randomized clinical trial (RCT) as the only valid method of clinical investigation, and it’s a symptom of the way that EBM relegates basic science knowledge, even well-established principles of science that show that something like, say, homeopathy or reiki is impossible under the current understanding of physics, chemistry and biology. However, never let it be said that RCTs aren’t actually important in SBM. Our problem with how EBM worships them derives from how it even bothers to do trials in the first place of modalities that can best be described by Harriet Hall’s brilliant appellation, Tooth Fairy Science. However, these days RCTs are widely perceived to have a serious problem. They have become so expensive to do and there have been so many failures of drugs that looked promising to show efficacy in clinical trials that some have even questioned whether there is something fundamentally wrong with how we do clinical trials now. Some even ask, as the title of an article by Clifton Leaf that appeared in the New York Times over the weekend, Do Clinical Trials Work?
It begins with the story of Avastin in brain tumors. I’m sure that Eric Merola will likely jump all over this, given how he tried to use the example of Avastin being approved for glioma on fast track approval that used phase II trials as the basis for doing so as an argument for why antineoplastons should be approved by the FDA. Or maybe he won’t. Here’s why. The story explains that there were two single-arm trials of adding Avastin to glioma therapy in which the tumors “shrank and the disease seemed to stall for several months when patients were given the drug.” Then Clifton points out the results of the randomized clinical trial presented at the American Society of Clinical Oncology (ASCO) meeting a month and a half ago:
But to the surprise of many, Dr. Gilbert’s study found no difference in survival between those who were given Avastin and those who were given a placebo.
Disappointing though its outcome was, the study represented a victory for science over guesswork, of hard data over hunches. As far as clinical trials went, Dr. Gilbert’s study was the gold standard. The earlier studies had each been “single-arm,” in the lingo of clinical trials, meaning there had been no comparison group. In Dr. Gilbert’s study, more than 600 brain cancer patients were randomly assigned to two evenly balanced groups: an intervention arm (those who got Avastin along with a standard treatment) and a control arm (those who got the latter and a placebo). What’s more, the study was “double-blind” — neither the patients nor the doctors knew who was in which group until after the results had been assessed.
The centerpiece of the country’s drug-testing system — the randomized, controlled trial — had worked.
This study could certainly be taken as evidence supporting a position that we shouldn’t approve drugs based on single-arm phase II clinical trials, even under fast track. It is indeed a very good example of how promising phase II clinical trial results are not always validated when the bigger and more rigorous phase III RCTs are performed. In one way, it is a good thing. Negative results, be they experimental or clinical trial, are just as important in science as positive results, if not more so. In another way, however, it’s a bad thing because, as the NYT article points out, “doctors had no more clarity after the trial about how to treat brain cancer patients than they had before.” A seemingly promising addition to the armamentarium against a deadly cancer that has too few effective treatments was shown not to work in an RCT that was designed to be, more or less, definitive. However, the key thing to remember about such an RCT is that it is looking at populations of patients. Overall, there was no difference in overall survival between the control and Avastin group, but that doesn’t necessarily mean that Avastin is useless against glioma.
Indeed, as someone who’s been studying angiogenesis and how to target it therapeutically in cancer since the heady days of the late 1990s, when findings by Judah Folkman and other pioneers in this field led to headlines in the lay press like “The Cure for Cancer” and it really did look as though the discovery that inhibiting angiogenesis produced dramatic results and outright cures in preclinical rodent models of cancer. Over the years, the study of angiogenesis has been gradually de-emphasized in my research, correlating inversely with the rise of other interests, but I do have a small project in targeting tumor-induced angiogenesis still ongoing and hope to publish on it before the end of the year. In any case, reality shut down those heady days, as it became clear that Avastin and other antiangiogenics were not as nontoxic in humans as they were in mice, nor were they nearly as effective. Still, it is clear that Avastin has contributed to significant increases in median survival in a number of tumor types, such as colorectal cancer. However, overall it’s hard not to conclude that antiangiogenic therapy has been, by and large, a disappointment, if only because the hype and hope were so sky-high 15 years ago. Rare indeed would have been the treatment that could have lived up to such expectations when tested in RCTs.
One thing that has been apparent for quite some time is that there appears to be a subset of patients who have remarkable responses to Avastin. Many oncologists get this feeling anecdotally, even if they don’t have evidence, and evidence has popped up in clinical trials. Assuming this is true, while it might not now make sense to treat all or most glioma patients with Avastin, it might very well make sense to treat that subset who have such dramatic responses if we could identify them beforehand. There’s the rub, though. We can’t, and Leaf points this out:
Some patients did do better on the drug, and indeed, doctors and patients insist that some who take Avastin significantly beat the average. But the trial was unable to discover these “responders” along the way, much less examine what might have accounted for the difference. (Dr. Gilbert is working to figure that out now.)
Indeed, even after some 400 completed clinical trials in various cancers, it’s not clear why Avastin works (or doesn’t work) in any single patient. “Despite looking at hundreds of potential predictive biomarkers, we do not currently have a way to predict who is most likely to respond to Avastin and who is not,” says a spokesperson for Genentech, a division of the Swiss pharmaceutical giant Roche, which makes the drug.
That we could be this uncertain about any medicine with $6 billion in annual global sales — and after 16 years of human trials involving tens of thousands of patients — is remarkable in itself. And yet this is the norm, not the exception. We are just as confused about a host of other long-tested therapies: neuroprotective drugs for stroke, erythropoiesis-stimulating agents for anemia, the antiviral drug Tamiflu — and, as recent headlines have shown, rosiglitazone (Avandia) for diabetes, a controversy that has now embroiled a related class of molecules. Which brings us to perhaps a more fundamental question, one that few people really want to ask: do clinical trials even work? Or are the diseases of individuals so particular that testing experimental medicines in broad groups is doomed to create more frustration than knowledge?
While it’s an excellent point that we don’t have predictive biomarkers (say, something in the blood we could measure) that tell us which patients are most likely to respond to Avastin (or most other drugs), Leaf seems to be indulging in a false dichotomy. Just because we don’t have predictive biomarkers for various drugs does not imply that clinical trials don’t work. Very clearly, they do. The problem is that they have limitations, and one of those limitations is that, without predictive biomarkers, we have no choice but to test the drug in a controlled population and see if there is a difference between control and the treated population that can be observed on a population level. The smaller the difference, the harder it is to detect and the more patients are needed to detect it. That’s why we need and want predictive biomarkers in the first place.
Worse, even the biomarkers we have are nowhere near 100% predictive. Let’s take a look at the prototypical targeted therapy, arguably the oldest targeted drug of all, Tamoxifen, which blocks estrogen activity. It is only used in tumors that make the estrogen receptor and are therefore presumed to be estrogen-responsive (i.e., estrogen stimulates them to grow). I remember a talk by the director of the Cancer Institute of New Jersey at the time I worked there, William Hait, who pointed out that Tamoxifen is effective in ER(+) cancers about 50% of the time. Around 70% of breast cancer is ER(+), and that means that if you treat all patients with breast cancer with Tamoxifen, you will see responses only 35% of the time, whereas if you treat only ER(+) cancers you will see responses 50% of the time. Another example is Herceptin, which targets amplified HER2 in breast cancer. Even though it is a targeted drug, it is effective against approximately 30% of HER2(+) cancers. Now, approximately 30% of breast cancers are HER2(+), which means that if you treat all comers with Herceptin, it will only be effective 0.3 x 0.3 = 0.09 (9%) of the time, but if you treat only HER2(+) cancers it should be effective 30% of the time. There are other examples he gave us. Taxol, for instance, is effective in 75% of breast cancer with p53 mutations. Since approximately 50% of breast cancers carry p53 mutations, if you treat all comers with Taxol you will get responses around 37.5% of the time, whereas if you treat only cancers with p53 mutations you should expect a 75% response rate. Of course, a 37.5% response rate is good enough that pretty much everyone with breast cancer who needs chemotherapy will get a Taxane, but you get the idea.
Now here’s where the devil is. These biomarkers that I’ve described are crude, and not even that predictive. But what, if anything, is better? That’s the problem, and that’s where most articles like this break down. They do an excellent job of identifying the problems with clinical trials, and there’s no doubt that Clifton Leaf does just that. None of these problems discussed in his NYT article are unfamiliar to most clinicians and clinical investigators, particularly in cancer. However, one notes that he has a book out entitled The Truth In Small Doses: Why We’re Losing the War on Cancer — and How to Win it. Personally, I hate that meme of “we’re losing the war on cancer,” because it’s not a war, and whether or not we’re “losing” depends on what your vision of “victory” is and how fast we can win the war. As I’ve pointed out many times, particularly around the 40th anniversary of Richard Nixon’s declaration of “war on cancer,” what do you expect in 40 years, given that the amount of resources we pour into this “war” are minuscule compared to what we spend on other things, such as—oh, you know—actual war? How much progress can we realistically expect in 40 years given that investment, the incredible complexity of cancer, and cancer’s ability to out-evolve almost anything we have as yet been able to throw at it. Clifton Leaf is a cancer survivor; so I can totally understand his frustration. However, that doesn’t stop his use of that tired old meme from irritating me. I’ll stop whining about that particular pet peeve of mine right now, but as everyone knows I do so love a good whine. Sorry.
My pet peeve aside, what can we do better? Most of us in oncology believe that the answer will likely come down to personalized medicine based on the genomic profile of each cancer, but how to get from the enormous amount of data from genomic studies of various cancer to actual validated treatments is not at all clear at this stage (other knowing that Stanislaw Burzynski’s doing it wrong). Right now, personalized medicine has a lot of promise but has even more hype with little or nothing as yet in the way of concrete results that clearly benefit patients. Many have been the ideas to overcome these problems and validate genomic-based personalized medicine. Leaf actually mentions an interesting one: The I-SPY2 TRIAL (Investigation of Serial Studies to Predict Your Therapeutic Response with Imaging And moLecular Analysis 2). (Whew, what a name!) It’s a very interesting prototype of how clinical trials might be done in the future, and if it works I can see a lot more trials like this:
The I-SPY 2 TRIAL (Investigation of Serial Studies to Predict Your Therapeutic Response with Imaging And moLecular Analysis 2) is a clinical trial for women with newly diagnosed locally advanced breast cancer to test whether adding investigational drugs to standard chemotherapy is better than standard chemotherapy alone before having surgery. The treatment phase of this trial will be testing multiple investigational drugs that are thought to target the biology of each participant’s tumor. The trial will use the information from each participant who completes the study treatment to help decide treatment for future women who join the trial. This will help the study researchers learn more quickly which investigational drugs will be most beneficial for women with certain tumor characteristics. The I-SPY 2 TRIAL will test the idea of tailoring treatment by using molecular tests to help identify which patients should be treated with investigational drugs. Results of this trial may help make investigational drugs available to more women in the future.
The beauty of this trial is that it uses Bayesian analysis of responses to have the trial, in effect, evolve in response to what is found at earlier stages. My main quibble with the study is that it requires that all subjects undergo pretreatment breast MRI before surgery, which has a tendency to upstage women through the Will Rogers effect and thus result in more mastectomies. I understand that the trial investigators probably wanted advanced imaging to follow tumor response and that MRI can also show blood flow and therefore measure tumor angiogenesis, but I always worry when I see a design like this one, that it might promote unnecessary mastectomies. On the other hand, the inclusion criteria require a tumor that is 2.5 cm in diameter or greater so perhaps this will be less of a problem. That quibble aside, as Leaf describes, it is an intriguing design and it does evolve based on previous results:
In fact, a breast cancer trial called I-SPY 2, already under way, may be a good model to follow. The aim of the trial, sponsored by the Biomarkers Consortium, a partnership that includes the Foundation for the National Institutes of Health, the F.D.A., and others, is to figure out whether neoadjuvant therapy for breast cancer — administering drugs before a tumor is surgically removed — reduces recurrence of the disease, and if so, which drugs work best.
As with the Herceptin model, patients are being matched with experimental medicines that are designed to target a particular molecular subtype of breast cancer. But unlike in other trials, I-SPY 2 investigators, including Dr. Berry, are testing up to a dozen drugs from multiple companies, phasing out those that don’t appear to be working and subbing in others, without stopping the study.
Here’s the design (more details can be found here and here, and some of the investigational drugs tried can be found here):
The difficult part of the study, of course, is designing the algorithms by which drugs are swapped out as they appear not to be working. If these decisions are made willy-nilly, then this trial would be no better than what Burzynski does (i.e., making simplistic guesses). However, there is a sophisticated analysis and algorithm by which treatment decisions are made. It does have to be remembered, though, that, although I-SPY2 does represent personalized medicine, it is not yet full genomic medicine. Most of the biomarker tests used are biomarkers that already exist, and the additional biomarkers measured will not affect patient treatment. This part of the trial is for discovery of biomarkers, not validation.
The bottom line
I’ll be watching the progress of I-SPY2 closely, because it’s a new kind of clinical trial. Whether it will succeed in improving the success of the followup clinical trials of agents identified through I-SPY remains to be seen, as it also remains to be seen whether it will speed up the pace of discovery. I’m probably less hopeful than Clifton Leaf, but that doesn’t mean I’m not hopeful.
So do clinical trials work? It depends on what you mean by “clinical trials” and “work.” I would argue that they do, in fact, still work in that they are still the best method we have to determine whether science-based therapies with preclinical promise actually translate into useful therapies. They’re simply evolving with science, as they must under the “selective pressure” of advances in technology and understanding of biology.

To debate or not to debate: The strange bedfellows of Andrew Weil
Those who stand on the wrong side of science love public debates and frequently challenge science advocates to them. The reason, of course, is that public debates are almost never a good venue to argue science, and these debates almost never happen in a truly neutral venue. This time around, I got a bit of ego gratification when Andrew Weil apparently wanted to have me do a debate. I reluctantly declined. Here’s why.
David Gorski on November 9, 2015
Link

To debate or not to debate, that is the question.Whether it is nobler in the mind to sufferThe slings and arrows of outrageous quackeryOr to take arms against a sea of quackademia,And, by opposing end them.
Hamlet, Act III, Scene 1, paraphrased badly.
The question of whether it is worthwhile to debate cranks, quacks, and advocates of pseudoscience has long been a contentious issue in the skeptic community. Those of you who’ve been reading my posts for a while know that I’ve always come down on the side that it is not a good idea One thing I’ve learned in my more than a decade of blogging, both here and at my not-so-super-secret other blog, is that advocates of pseudoscience love public debates. Indeed, whenever you see a skeptic agree to a public debate with an advocate of pseudoscience, it’s a damned sure bet that it wasn’t the skeptic who proposed it. I suppose it’s possible that there have been such instances that I’m unaware of, but I do know of a lot of instances where it was the other way around. I’ve even witnessed one myself, when our fearless founder Steve Novella debated antivaccine quack Julian Whitaker about vaccine safety at FreedomFest in Las Vegas while we were at TAM three years ago. Steve mopped the floor with Dr. Whitaker so dramatically that it almost changed my mind about the value of debates with quacks because, witnessing the debate, I saw that the arguments Dr. Whitaker marshaled were such hackneyed antivaccine talking points that I knew I could also have demolished them. Still, in the end, no minds were likely to be changed, and the question of vaccine safety was clearly being used as a tool to oppose school vaccine mandates or, as antivaccinationists like to call them deceptively, “forced vaccination.” Whether vaccines are safe and effective or not is a separate question from whether the government should mandate certain vaccines as a precondition for attending school or being in day care.
Over the years, I myself have been “challenged” to similar debates myself. Perhaps the most bizarre example occurred when someone claiming to represent HIV/AIDS denialist Christine Maggiore contacted me claiming that she wanted to arrange a debate between us. Maggiore, unfortunately, died a mere two years later of—you guessed it—AIDS-related complications. Although occasionally the ego gratification of being asked to participate in such events vied with my longstanding belief that debating cranks doesn’t sway anyone, sharing the stage with a real scientist does unduly elevate the crank in the eyes of the public. Besides, whatever the seeming outcome of the debate, you can count on the crank to declare victory and his believers to agree. In any event, science isn’t decided by the metrics used to judge who “wins” a public debate, which rely more on rhetoric and cleverness rather than science to decide the outcome. Finally, such debates are not without risks. Although Julian Whitaker, for example, was terrible at it, other cranks are adept at the Gish Gallop, and an unprepared skeptic or scientist can be made to appear clueless in front of a crowd that is almost always packed with supporters of the crank, not the skeptic.
Just last week, there was another “debate” challenge that led me to question my resolve not to debate cranks. It came from a most unexpected source.
In which I am challenged to a public debate
About a week ago, I checked my email and was surprised to find a message from John Jackson, who identified himself as the Executive Director of the Adolph Coors Foundation, the charitable arm of the Coors family:
I am writing to invite your participation in a debate on integrative medicine which will be held Sunday evening, March 20, 2016, at the Hyatt Regency in Denver. The debate will be the keynote event of our Pioneers in Health conference. Your debate partner will be Dr. Andrew Weil, who has confirmed participation. We expect excellent attendance of 700-800, possibly more. At least a portion of the audience will include those attending Dr. Weil’s annual Nutrition and Health Conference which begins the following morning. We also expect our conference cosponsor, Americans for Prosperity Foundation (AFPF), to attract attendees through their outreach efforts. AFPF is a grassroots organization that has virtually nothing to do with any kind of medicine, conventional or integrative. AFPF’s interest is promoting innovations in the delivery of health care (more health care choice) which will be the focus of a panel earlier in the day.
The debate topic: “Fad or the Future: Will Integrative Medicine Play a Growing Role in the Future of Health Care?”
I have read numerous articles you have authored on the SBM website and feel your thoughtful and well-reasoned posts (on a whole host of issues) offer an excellent counterpoint to Dr. Weil. Our ground rules for debate are simple: We insist on respect and politeness for differing points of view. Respectful, open and free debate has become somewhat of a rarity in our highly polarized society. However, we feel it is still possible to have smart people discuss important issues in a way that informs and enlightens.
If you are willing to join us, we plan the following debate format: You both make opening statements, you both respond to each other’s opening statements, you ask each other questions, you both answer questions from the moderator and the audience and, finally, you both will be given equal time to offer closing statements. Our foundation will select the moderator which will be a neutral journalist and/or business leader. FYI, audience participation will be limited to questions submitted via a mobile app, not by microphone.
In the interest of full disclosure, our foundation funds several institutions which are studying various “alternative” practices, including the recent study of the use of electro-acupuncture for hypertension. We have also funded a project involving Dr. Weil and the University of Arizona. You can read more about our interest in integrative medicine and who we fund at www.coorsfoundation.org. We are not going to be offended if you say something critical about our focus or if you offer suggestions on ways we can improve it.
Of course, if you agree to participate, we would cover your travel, meals and lodging expenses and are willing to negotiate a reasonable honoraria.
If you would like to discuss this further by phone, please feel free to give me a call. I can be reached in Denver at 303-[REDACTED].
Thank you, Dr. Gorski. I look forward to your response.
John JacksonAdolph Coors FoundationDenver
I found this offer extremely tempting, but, I must confess, for the wrong reason. As most of you know, Andrew Weil is a big cheese in the world of “integrating” alternative medicine quackery with real medicine. He’s nationally and internationally regarded as a pioneering expert in “complementary and alternative medicine” (CAM). Indeed, arguably no human being has done more in his lifetime and career to seemingly legitimize the specialty of “integrative medicine.” He’s even started an integrative medicine residency and managed to spearhead a board certification in integrative medicine, although the latter does not use the usual accrediting organization used by most other established medical specialties. Interestingly, many CAM practitioners were not pleased by Weil’s original proposal because the board certification is only available to physicians.
Yes, I’m a bit ashamed to admit that it was primarily the ego massage that Mr. Jackson’s offer provided that led to my waffling, which continued over the course of a few days. I ignored (for the moment) the red flags present in the very invitation, including the fact that the audience would almost certainly be packed with Weil supporters arriving early to attend Weil’s conference the next day and, also as disturbing, the involvement of Americans for Prosperity Foundation, which is a right-wing foundation funded by the billionaire Koch brothers.
During that time, I e-mailed a bunch of my fellow SBM contributors and a few others for opinions and received conflicting advice. One camp was divided into the “Do it!” mode; the other telling me it would be a bad idea. Oddly enough, the first camp was definitely larger than the second. What I also learned was that Mr. Johnson had asked Edzard Ernst first, who had declined. If my ego was stoked before by being chosen to be invited to debate the foremost practitioner of quackademic medicine in the world, then being a second choice after Edzard Ernst, rather than deflating my ego at being second choice, puffed it up even more, given my admiration for Prof. Ernst. Ernst also blogged about it in a post he entitled Standing up for science? Yes, but not at any cost!, in which he declined the public debate and offered to conduct a written debate on his blog. It’s similar to what I proposed when an antivaccine group challenged us to have two of us debate two antivaccinationists publicly. The antivaccine group declined. Likewise, Mr. Jackson declined Prof. Ernst’s offer as well, given that he never wrote back.
My personal response, after having waffled for a while, requested unedited video as a minimal precondition, discussed live streaming the event briefly by email, and failed to find time during business hours to telephone Mr. Jackson for two days, was similar. After overcoming my ego gratification and conflicting advice weighted heavily towards doing something that I didn’t believe in and didn’t think to be a good tactic, I finally emailed Mr. Jackson and stated that I would not debate Dr. Weil in the format proposed. I said that I would be happy to give a standalone talk, but I would not debate. Three days later, I, too, have not heard back from Mr. Jackson. So I have something else in common with Ernst!
Ernst’s post concluded with two questions that I will address in the remainder of this post:
    • Should we stand up for science wherever we can, or is the price occasionally simply too high?
    • What are these mysterious links between alternative medicine in the US and the far right?
“All truth comes from public debate”: A quack manifesto
In addressing Prof. Ernst’s first question with respect to whether we as supporters of science should accept offers to debate pseudoscientists like Andrew Weil, let me discuss briefly why it might be that cranks, quacks, and pseudoscientists find live public debates so attractive. Indeed, I’ve referred to a seemingly near-inviolable belief among pseudoscience promoters that “all truth comes from live public debate.” It’s a belief that I even tried to express using rudimentary Latin as omne verum est a forensem principle. (Latin sounds so much more cool for this, but I have no idea whether this is the best translation—or even grammatically correct; maybe Latin scholars out there can suggest better.) For instance, antivaccine guru Andrew Wakefield challenged Dr. David Salisbury to a “live public debate” about whether the MMR vaccine causes autism or not. (Hint to Wakefield: It doesn’t.) Other examples regular readers might remember through the years include the aforementioned Julian Whitaker debating Steve Novella; Michael Shermer’s “debate” with Deepak Chopra; and antivaccine propagandist David Kirby debating author Arthur Allen. To reiterate, I don’t “debate” cranks, at least not live on stage in such artificial events, because such events (1) make it appear that there is an actual scientific debate when there is not and (2) give the crank the freedom to Gish Gallop to his or her heart’s content.
There’s actually a third reason. The call for a “live public debate” is also an intentional strategy.
A couple of years ago, when I was in the thick of trying to refute the cancer cure claims of Stanislaw Burzynski (who, by the way, is scheduled on November 19 to go to court against the Texas Medical Board, which is trying to do something long overdue and strip him of his medical license), I noticed a series of Tweets (now gone) from Burzynski supporters challenging occasional SBM blogger Peter Lipson and/or “The Skeptics™” to a live debate:
“The SkeptiCowards”couldn’t handle a live DebateThey believe in “Free Speech”as long as it’s their speech-that takes the Cake!#Burzynski
— QbertQbert (@QbertQbert) April 26, 2013
Elsewhere, on a public pro-Burzynski group called the Burzynski Facebook Patient Group, a man named Randy Hinton also called out Peter Lipson on a now-closed Burzynski patient Facebook group:
The only way to ever successfully deal with a growing number of medical mafia internet propaganda minister’s [sic] trying to smear and undermind [sic] Stanislaw Burzynski is to do exactly what I have tried to do for the last 24 hour’s [sic]. Call these maggot’s [sic] out and publically dare them to debate the topic of ANP verses chemotherapy in front of a large live audience with no restriction’s [sic] or sensoring [sic] of information or statement’s [sic]. Multiple attempt’s [sic] yesterday to get Peter Lipson who wrote that pile of crap in Forbes to agree to this got virtually no response. The same will [sic] true of the other pharmawhores doing the same thing all across the country right now. These INTERNET KEYBORAD [sic] GOBLIN’S [sic] will never agree to it because they cann’not [sic] CONTROL the conversation the way they do on their blog’s [sic]. CALL THEM OUT TO A PUBLIC DEBATE AT EVERY TURN.
Later in the comments after his post, Mr. Hinton says:
I have got money that say’s [sic] they run from a live debate like scared rabbit’s [sic]. I have told Eric he need’s [sic] to CALL THEM OUT!!!
Mr. Hinton was obviously referring to Eric Merola, Stanislaw Burzynski’s propagandist, who has in the past unwisely characterized a skeptical blogger near and dear to my heart as a white supremacist and someone too busy eating puppies to bother to read the scientific literature about Burzynski.
These two Burzynski supporters demonstrate what I’m talking about. They seem to think that science is decided in public debates and view the quite proper reluctance among scientists like myself and skeptics to engage cranks in such spectacles as “cowardice.” It is not, but cranks continue to labor under the delusion that science is somehow decided in such forums, which are a variant of a sort of argumentum ad populum, in which something is argued to be true because it is popular or, in a debate, an argument is thought to be closer to the truth because it is more popular. Science doesn’t work that way. It is decided on evidence presented at scientific conferences and in peer-reviewed journals, where the real scientific debate plays out until it is temporarily settled and scientists come to a provisional consensus. That provisional consensus, of course, is always subject to change as new observations, data, and experimental results come to light, but it takes observations, data, and experimental results to change the consensus, not “live public debates.” Such “live public debates” are meant for one thing and one thing only: To sway public opinion to a viewpoint not supported by science, in the process elevating pseudoscience or the unproven to the same plane as the scientific consensus as a scientifically viable “alternative.” It’s also a no-lose proposition most of the time. If the scientist declines, the pseudoscientist can cry, “Coward!” If he accepts, no matter what the outcome, few, if any, minds are changed and the pseudoscientist has been seen on stage with a real scientist.
Does that rule apply here, though? After all, most likely it was not Andrew Weil who was pushing for this debate, although he obviously must have agreed to it first, as he is still listed as being featured on the site, as is the debate. Also, although Andrew Weil and Edzard Ernst can be viewed as being roughly of the same stature within their respective sides of the quackademic/science-based medicine divide, pivoting to invite me clearly involved a massive drop-off in stature for Weil’s opponent. Although I might be somewhat well-known in the skeptical and medical blogosphere, I’m no equivalent to Weil in the world of woo, which is why I’m surprised they didn’t approach, for example, Steve Novella first after Prof. Ernst declined their offer, especially since he’s done debates like this before and I have not. On the other hand, maybe that’s the reason. Steve is a known quantity and that quantity is known to be quite good at these debates.
One thing that is true, however, is that the Adolph Coors Foundation is well known for promoting integrative medicine. Mr. Jackson said as much himself in his email, and the Coors Foundation website touts its projects with the Center for Integrative Medicine at the University of Arizona (run, of course, by Andrew Weil), the Susan Samueli Center for Integrative Medicine at UC Irvine, and the Myrna Brind Center for Integrative Medicine at Jefferson Hospital. For instance, the project at UA promises:
The health insurance industry, and its practice of not reimbursing doctors who prescribe CAM-type treatments, creates a massive bottleneck that prevents promising integrative treatments from entering the medical mainstream. The Adolph Coors Foundation has awarded a gift to the Center for Integrative Medicine at the University of Arizona to support a three-year study of the effect of integrative medicine on health care and health costs.
Specifically, the study will generate a statistically relevant comparison of the health and cost outcomes of two populations of Maricopa County, Arizona, employees … one whose primary care is delivered using an integrative model and the other whose primary care is delivered using a traditional conventional care model. The outcomes generated by this study will be shared with a broad range of medical and health professionals, insurance companies, academics, business leaders and policymakers.
In other words, the Coors Foundation is funding a study at Weil’s Center to “prove” that integrative medicine produces better outcomes and thus persuade the health insurance industry to pay for quackery. Interestingly, this study, the Integrative Medicine PrimAry Care Trial (IMPACT, ClinicalTrials.gov identifier NCT01785485) was started in 2013, and its record hasn’t been updated since then. Enrollment was supposed to have been completed in October 2015, but the record as of 2013 indicates that not a single patient has yet been accrued. Its protocol was published in April 2014 and looks to be measuring so many outcomes that it’s virtually certain that one of them will be positive. So what’s going on with this study? Who knows? There’s not even much about it on the University of Arizona website. It’s a lot harder to find out online what’s going on with a study funded by a private entity than it is for a federally-funded study.
The Samueli Center project involves using electroacupuncture (which, of course, isn’t really acupuncture) to treat hypertension, while the Brind Center project is looking at high dose N-Acetyl-Cysteine as a treatment for breast and prostate cancer. So, yes, the Coors Foundation is deep into funding quackademic medicine and thus has a definite interest in promoting the acceptance of “integrative medicine.”
Although I highly doubt Mr. Jackson had any ulterior motives in trying to organize this debate, he does run a foundation that promotes political policies the Coors family supports. So he likely thinks more in political rather than scientific terms. In politics, a debate is how politicians try to persuade; in science, not so much. It’s also possible that he just thought it would be an entertaining way to discuss integrative medicine. Entertaining it might have been, but public debates of this sort, with their emphasis on rhetoric over substance and the ease with which one can Gish Gallop, are rarely particularly informative. They can serve as entertainment. They can serve as propaganda. Most often they serve as both. But they are seldom a good way to effectively communicate science over pseudoscience, medicine over quackery.
Strange bedfellows: Integrative medicine, Andrew Weil, and the American right
Prof. Ernst appeared quite surprised to learn of the right wing proclivities of the Coors Foundation and the Americans for Prosperity Foundation, apparently because he associated alternative medicine more with the political left. He shouldn’t have been. I’ve pointed out many times over the years at all my blogging locales how the idea that alternative medicine is mainly the purview of hippy dippy leftists and refugees from the 1960s counterculture like Andrew Weil is largely a myth. Sure, there are a lot of alt-med mavens like that, and, yes, the most influential member of the integrative medicine movement is the aforementioned Andrew Weil. However, there is also a lesser known but very powerful component to the alt-med movement on the right. A lot of the reason for this is financial. The supplement industry in the US is concentrated in Utah, and it’s no surprise that its lapdogs in Congress (such as Senator Orrin Hatch and Representative Jason Chaffetz) are pretty much all Republican. More importantly, the conspiratorial bent of alt-med believers meshes well with that of a significant segment of the far right.
Indeed, if you look at the “health freedom” movement (which includes many supporters of alternative medicine and, in particular supplements), you will find that it is very libertarian/conservative-leaning, mainly because to this movement the FDA (whose mission is to make sure medicines marketed in the US are safe and effective) and state medical boards (whose mission is to license and regulate the practice of medicine) are the enemy. After all, the FDA and the FTC prevent quacks from using whatever unproven medicines they want and making whatever health claims they want about those medicines, and state medical boards, when not too busy sanctioning physicians with substance abuse problems, do sometimes try to take medical licenses away from quacks, as difficult as that all too often turns out to be. So the anti-government bent of quacks and their supporters meshes well with the antiregulatory tendencies of the right wing.
Thus, unlike the case with Prof. Ernst, it was not much of a surprise to me to discover that the Coors Foundation and the Americans for Prosperity (AFP) Foundation are co-sponsoring an event featuring Andrew Weil. AFP was founded by the billionaire Koch brothers and, although the AFP and AFP Foundation are separate entities, they are “joined at the hip,” so to speak and frequently work in parallel. In any case, the AFP Foundation is anything but a “grassroots organization” (as it was described by Mr. Jackson). Rather, AFP and the AFP Foundation are the tools through which the Koch brothers exercise much of their political influence and AFP was intimately linked to the rise of the Tea Party. Indeed, the two worked together since the very inception of the Tea Party in 2009. It has even been alleged that front groups with ties to the tobacco industry and Koch brothers planned the creation of the Tea Party a decade before its emergence. Whatever you think of the Tea Party, there is little doubt that AFP is a very conservative/libertarian organization. Among other conservative causes, AFP has vociferously opposed the Affordable Care Act and been a major force spreading anti-science misinformation about anthropogenic global climate change. Funding and promoting anti-science is what AFP and the AFP Foundation do; so I’m not too surprised that the AFP Foundation would be involved in funding a conference like the one I was invited to.
But what about the Coors Foundation? I knew much less about it when I was first contacted, but what I’ve learned about it suggests it’s problematic as well, for many of the same reasons. For example:
In the early 1970s Coors required prospective employees to submit to a lie-detector test in which the company asked if the respondent was a homosexual (prompting Harvey Milk to organize a boycott of Coors beer). Throughout the 1970s and ’80s, the Coors family used generous donations from the Adolph Coors Foundation to launch right-wing groups like the Moral Majority, the Heritage Foundation and the Free Congress Foundation–which coined the term “the homosexual agenda” in publications like Gays, AIDS and You. Meanwhile, the company busted unions (leading to an AFL-CIO boycott), and individual family members like William Coors made racist speeches to black audiences claiming “one of the best things that they [slave traders] did for you was to drag your ancestors over here in chains” (this statement led to a boycott by numerous minority groups).
As a result of protests, the Coors Foundation in 1993 restricted its philanthropy to Colorado and formed the Castle Rock Foundation, which continued to fund far right groups and causes. In 2011, Castle Rock was absorbed back into the Coors Foundation. In any case, one group the Coors Foundation funded, the Heritage Foundation, is well-known for denying anthropogenic climate change and opposing efforts to mitigate it. These days, the Coors Foundation’s priorities sound relatively benign on the surface compared to its past activities (although one notes that some of its priorities are identical to the mission of the Castle Rock Foundation). Be that as it may, the Coors Foundation is clearly very conservative in its orientation and it’s taken on as one of its major focuses the promotion of the “integration” of quackery with medicine.
Integrative medicine, it’s not just for hippy-dippy lefties any more—if it ever was in the first place. It’s now made for some very strange bedfellows.
Aftermath: Ego and money lose to science
As I close, I must admit that I’m left wondering whether I made the right decision or not. I’d be lying if I didn’t admit that it could well have been very cool to be (hopefully) the hero defending science against Andrew Weil, the way that Bill Nye defended evolution against Ken Ham at the Creation Museum in Kentucky. But then I consider again my longstanding belief that public debates are not a good strategy to defend science, coupled with a cold, hard assessment of the cost-benefit ratio. To my mind, the cost in terms of preparation and the associations drawn to me by doing an event linked to groups like the Americans For Prosperity Foundation and the Coors Foundation were just not worth the (at most) modest potential benefit of a win (assuming I could even “win”) for science-based medicine. I briefly toyed with the idea of asking for the same honorarium that they were paying Dr. Weil (which is no doubt significant, likely way more than I’ve ever been paid before to give a talk) as a means of seeing just how Mr. Jackson wanted this debate to take place, but such is not my way and I quickly dropped that idea.
The bottom line is that no amount of money I could reasonable expect, even if the Coors Foundation agreed to pay me as much as they are paying Weil, or ego gratification derived from being on the same stage with the godfather of “integrative medicine” would be worth the effort and risk of doing a debate like this one. Far more importantly, there was no way I could see such a debate being more than a wash or, at best, a modest “win” for science-based medicine with the risk of its being a big loss. In the end, I made my decision, and I’m OK with it. Others might make a different one.

   Lies, damned lies, and…science-based medicine?
David Gorski
Link
I realize that in the question-and-answer session after my talk at the Lorne Trottier Public Science Symposium a week ago I suggested in response to a man named Leon Maliniak, who monopolized the first part of what was already a too-brief Q&A session by expounding on the supposed genius of Royal Rife, that I would be doing a post about the Rife Machine soon. And so I probably will; such a post is long overdue at this blog, and I’m surprised that no one’s done one after nearly three years. However, as I arrived back home in the Detroit area Tuesday evening, I was greeted by an article that, I believe, requires a timely response. (No, it wasn’t this article, although responding to it might be amusing even though it’s a rant against me based on a post that is two and a half years old.) Rather, this time around, the article is in the most recent issue of The Atlantic and on the surface appears to be yet another indictment of science-based medicine, this time in the form of a hagiography of Greek researcher John Ioannidis. The article, trumpeted by Tara Parker-Pope, comes under the heading of “Brave Thinkers” and is entitled Lies, Damned Lies, and Medical Science. It is being promoted in news stories like this, where the story is spun as indicating that medical science is so flawed that even the cell-phone cancer data can’t be trusted:
Visit msnbc.com for breaking news, world news, and news about the economy
Let me mention two things before I delve into the meat of the article. First, these days I’m not nearly as enamored of The Atlantic as I used to be. I was a long-time subscriber (at least 20 years) until last fall, when The Atlantic published an article so egregiously bad on the H1N1 vaccine that our very own Mark Crislip decided to annotate it in his own inimitable fashion. That article was so awful that I decided not to renew my subscription; it is to my shame that I didn’t find the time to write a letter to The Atlantic explaining why. Fortunately, this article isn’t as bad (it’s a mixed bag, actually, making some good points and then undermining some of them by overreaching), although it does lay on the praise for Ioannidis and the attacks on SBM a bit thick. Be that as it may, clearly The Atlantic has developed a penchant for “brave maverick doctors” and using them to cast doubt on science-based medicine. Second, I actually happen to love John Ioannidis’ work, so much so that I’ve written about it at least twice over the last three years, including The life cycle of translational research and Does popularity lead to unreliability in scientific research?, where I introduced the topic using Ioannidis’ work. Indeed, I find nothing at all threatening to me as an advocate of science-based medicine in Ioannidis’ two most famous papers, Contradicted and Initially Stronger Effects in Highly Cited Clinical Research and Why Most Published Research Findings Are False. The conclusions of these papers to me are akin to concluding that water is wet and everybody dies. It is, however, quite good that Ioannidis is there to spell out these difficulties with SBM, because he tries to keep us honest.
Unfortunately, both papers are frequently wielded like a shibboleth by advocates of alternative medicine against science-based medicine (SBM) as “evidence” that it is corrupt and defective to the very core and that therefore their woo is at least on equal footing with SBM. Ioannidis has formalized the study of problems with the application of science to medicine that most physicians intuitively sense but have not ever really thought about in a rigorous, systematic fashion. Contrast this to so-called “complementary and alternative medicine” (i.e., CAM), where you will never see such a questioning of the methodology and evidence base behind it (mainly because its methodology is primarily anecdotal and its evidence base nonexistent or fatally flawed) and most practitioners never change their practice as a result of any research, and you’ll see my point.
Right from the beginning, the perspective of the author David H. Freedman is clear. I first note the title of the article (Lies, Damned Lies, and Medical Science) is intentionally and unnecessarily inflammatory. On the other hand, I suppose that entitling it something like “Why science-based medicine is really complicated and most medical studies ultimately turn out to be wrong” wouldn’t have been as eye-catching. Even Ioannidis restrained himself more when he entitled his PLoS review an almost as exaggerated Why Most Published Research Findings Are False, which has made it laughably easy for cranks to the misuse and abuse of his article. My annoyance at the title and general tone of Freedman’s article notwithstanding, coupled with the sorts of news coverage it’s getting notwithstanding, there are still important messages in Freedman’s article worth considering, if you get past the spin, which begins very early in describing Ioannidis and his team thusly:
Last spring, I sat in on one of the team’s weekly meetings on the medical school’s campus, which is plunked crazily across a series of sharp hills. The building in which we met, like most at the school, had the look of a barracks and was festooned with political graffiti. But the group convened in a spacious conference room that would have been at home at a Silicon Valley start-up. Sprawled around a large table were Tatsioni and eight other youngish Greek researchers and physicians who, in contrast to the pasty younger staff frequently seen in U.S. hospitals, looked like the casually glamorous cast of a television medical drama. The professor, a dapper and soft-spoken man named John Ioannidis, loosely presided.
I’m guessing the only reason Freedman didn’t liken this team to Dr. Greg House and his minions is because, unlike Dr. House, Ioannidis is dapper and soft-spoken, although like Dr. House’s team apparently Ioannidis’ team is full of good-looking young doctors. After describing how Ioannidis delved into the medical literature and was shocked by the number of seemingly important and significant published findings that were later reversed in subsequent studies, Freedman boils down the what I consider to be the two most important messages that derive from Ioannidis’ work:
This array suggested a bigger, underlying dysfunction, and Ioannidis thought he knew what it was. “The studies were biased,” he says. “Sometimes they were overtly biased. Sometimes it was difficult to see the bias, but it was there.” Researchers headed into their studies wanting certain results—and, lo and behold, they were getting them. We think of the scientific process as being objective, rigorous, and even ruthless in separating out what is true from what we merely wish to be true, but in fact it’s easy to manipulate results, even unintentionally or unconsciously. “At every step in the process, there is room to distort results, a way to make a stronger claim or to select what is going to be concluded,” says Ioannidis. “There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded.”
Perhaps only a minority of researchers were succumbing to this bias, but their distorted findings were having an outsize effect on published research. To get funding and tenured positions, and often merely to stay afloat, researchers have to get their work published in well-regarded journals, where rejection rates can climb above 90 percent. Not surprisingly, the studies that tend to make the grade are those with eye-catching findings. But while coming up with eye-catching theories is relatively easy, getting reality to bear them out is another matter. The great majority collapse under the weight of contradictory data when studied rigorously. Imagine, though, that five different research teams test an interesting theory that’s making the rounds, and four of the groups correctly prove the idea false, while the one less cautious group incorrectly “proves” it true through some combination of error, fluke, and clever selection of data. Guess whose findings your doctor ends up reading about in the journal, and you end up hearing about on the evening news? Researchers can sometimes win attention by refuting a prominent finding, which can help to at least raise doubts about results, but in general it is far more rewarding to add a new insight or exciting-sounding twist to existing research than to retest its basic premises—after all, simply re-proving someone else’s results is unlikely to get you published, and attempting to undermine the work of respected colleagues can have ugly professional repercussions.
Of course, we’ve discussed the problems of publication bias before multiple times right here on SBM. Contrary to the pharma conspiracy-mongering of many CAM advocates, more commonly the reason for bias in the medical literature is what is described above: Simply confirming previously published results is not nearly as interesting as publishing something new and provocative. Scientists know it; journal editors know it. In fact, this is far more likely a problem than the fear of undermining the work of respected colleagues, although I have little doubt that that fear is sometimes operative. The reason is, again, because novel and controversial findings are more interesting and therefore more attractive to publish. A young investigator doesn’t make a name for himself by simply agreeing with respected colleagues. He makes a name for himself by carving out a niche and even more so if he shows that commonly accepted science has been wrong. Indeed, I would argue that this is the very reason that comparative effectiveness research (CER) is given such short shrift in the medical literature, so much so that the government has decided to encourage it in the latest health insurance reform bill. CER is nothing more than comparing already existing and validated therapies head-to-head against each other to see which is more effective. To most scientists, nothing could be more boring, no matter how important CER actually is. Until recently, doing CER was a good way to bury a medical academic career in the backwaters. Hopefully, that will change, but to my mind the very problems Ioannidis points out are part of the reason why CER has had such rough sledding in achieving respectability.
More importantly, what Freedman appears (at least to me) to portray as a serious, nigh unfixable problem in the medical research that undergirds SBM is actually its greatest strength: it changes with the evidence. Yes, there is a bias towards publishing striking new findings and not publishing (or at least not publishing in highly prestigious journals) less striking or negative findings. This has been a well-known bias that’s been bemoaned for decades; indeed, I remember learning about it in medical school, and you don’t want to know how long ago I went to medical school.
Even so, Freedman inadvertently echoes a message that we at SBM have discussed many times, namely that high quality evidence is essential. In the article, Freedman points out that 80% of nonrandomized trials turn out to be wrong, as are “25 percent of supposedly gold-standard randomized trials, and as much as 10 percent of the platinum-standard large randomized trials.” Big surprise, right? Less rigorous designs produce false positives more often! Also remember, in an absolutely ideal world with a perfectly designed randomized clinical trial (RCT), by choosing p<0.05 as the cutoff for statistical significance, we would expect that at least 5% of RCTs will be wrong by random chance alone. Add type II errors to that and the number is expected to be even higher, again, just by random chance alone. When you consider these facts, then having only 10% of large randomized trials turn out to be incorrect is actually not too bad at all. Even if only 25% of all randomized trials turn out to be wrong, that isn’t all that bad either; these include smaller trials. After all, the real world is messy; trials are never perfect, nor is their analysis. The real messages should be that lesser quality trials that are unrandomized are highly unreliable and that even randomized trials should be replicated if at all possible. Unfortunately, resources are such that such trials can’t always be replicated or expanded upon, which means that we as scientists need to do our damnedest to work on improving the quality of such trials. Also, don’t forget that the probability of a trial being wrong increases as the implausibility of the hypothesis being tested increases, as Steve Novella and Alex Tabarrok have pointed out in discussing Ioannidis’ results. Unfortunately, with the rise of CAM, more and more studies are being done on highly implausible hypotheses, which will make the problem of false-positive studies even worse. Is this contributing to the problem overall? I don’t know, but that would be a really interesting hypothesis for Ioannidis and his group to study, don’t you think?
Another important lesson from Ioannidis’ work cited by Freedman is that hard outcomes are much more important than soft outcomes in medical studies. For example, death is the hardest outcome of all. If a treatment for a chronic condition is going to claim benefit, it behooves researchers to demonstrate that it has a measurable effect on mortality. I discussed this issue a bit in the context of the controversy over Avastin and breast cancer, where the RCTs used to justify approving Avastin for use against stage IV breast cancer found an effect on disease-free survival but not overall survival. However, this issue is not important just in cancer trials, but in any trial for an intervention that is being used to reduce mortality. “Softer” outcomes, be they disease-free survival, reductions in blood lipid levels, reductions in blood pressure, or whatever, are always easier to demonstrate than decreased mortality.
Unfortunately, one thing that comes through in Freedman’s article is something similar to other work I’ve seen from him. For instance, when Freedman wrote about Andrew Wakefield back in May, he got it so wrong that he was not even wrong when he described The Real Lesson of the Vaccines-Cause-Autism Debacle. To him the discovery of Andrew Wakefield’s malfeasance is as nothing compared to what he sees as the corruption and level of error present in the current medical literature. In other words, Freedman presented Wakefield not as a pseudoscience maven, an aberration, someone outside the system who somehow managed to get his pseudoscience published in a respectable medical journal and thereby caused enormous damage to vaccination programs in the U.K. and beyond. Oh, no. To Freedman, Wakefield is representative of the system. One wonders, given how much he distrusts the medical literature, Freedman actually knew Wakefield was wrong. After all, all the studies that refute Wakefield presumably suffer from the same intractable problems that Freedman sees in all medical literature. In any case, perhaps this apparent view explains why, while Freedman gets some things right in his profile of Ioannidis, he gets one thing enormously wrong:
Ioannidis initially thought the community might come out fighting. Instead, it seemed relieved, as if it had been guiltily waiting for someone to blow the whistle, and eager to hear more. David Gorski, a surgeon and researcher at Detroit’s Barbara Ann Karmanos Cancer Institute, noted in his prominent medical blog that when he presented Ioannidis’s paper on highly cited research at a professional meeting, “not a single one of my surgical colleagues was the least bit surprised or disturbed by its findings.” Ioannidis offers a theory for the relatively calm reception. “I think that people didn’t feel I was only trying to provoke them, because I showed that it was a community problem, instead of pointing fingers at individual examples of bad research,” he says. In a sense, he gave scientists an opportunity to cluck about the wrongness without having to acknowledge that they themselves succumb to it—it was something everyone else did.
To say that Ioannidis’s work has been embraced would be an understatement. His PLoS Medicine paper is the most downloaded in the journal’s history, and it’s not even Ioannidis’s most-cited work—that would be a paper he published in Nature Genetics on the problems with gene-link studies. Other researchers are eager to work with him: he has published papers with 1,328 different co-authors at 538 institutions in 43 countries, he says. Last year he received, by his estimate, invitations to speak at 1,000 conferences and institutions around the world, and he was accepting an average of about five invitations a month until a case last year of excessive-travel-induced vertigo led him to cut back.
Yes, my ego can’t resist mentioning that I was quoted in Freedman’s article. My ego also can’t help but be irritated that Freedman gets it completely wrong in how he spins my anecdote. Instead of the interpretation I put on it, namely that physicians are aware of the problems in the medical literature described by Ioannidis and take such information into account when interpreting studies (i.e., that Ioannidis’ work is simply reinforcement of what they know or suspect anyway), Freedman instead interprets my colleagues’ reaction to Ioannidis as “an opportunity to cluck about the wrongness without having to acknowledge that they themselves succumb to it—it was something everyone else did.” I suppose it’s possible that there is a grain of truth in that — but only a small grain. In reality, at least from my observations, the reason that scientists and skeptics have not only refrained from attacking Ioannidis but in actuality have embraced him and his findings of deficiencies in how we do clinical trials is for the right reasons. We want to be better, and we are not afraid of criticism. Try, for instance, to imagine an Ioannidis in the world of CAM. Pretty hard, isn’t it? Then picture how a CAM-Ioannidis would be received by CAM practitioners? I bet you can’t imagine that they would shower him with praise, publications in their best journals, and far more invitations to speak at prestigious medical conferences than one person could ever possibly accept.
Yet that’s how science-based practitioners have received John Ioannidis.
In the end, Ioannidis has a message that is more about how little the general public understands the nature of science than it is about the flaws in SBM:
We could solve much of the wrongness problem, Ioannidis says, if the world simply stopped expecting scientists to be right. That’s because being wrong in science is fine, and even necessary—as long as scientists recognize that they blew it, report their mistake openly instead of disguising it as a success, and then move on to the next thing, until they come up with the very occasional genuine breakthrough. But as long as careers remain contingent on producing a stream of research that’s dressed up to seem more right than it is, scientists will keep delivering exactly that.
“Science is a noble endeavor, but it’s also a low-yield endeavor,” he says. “I’m not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life. We should be very comfortable with that fact.”
We should indeed. On the other hand, those of us in the trenches with individual patients don’t have the luxury of ignoring many studies that conflict (as Ioannidis suggests elsewhere in the article). Moreover, it is science that gives us our authority with patients. If patients lose trust in science, then there is little reason not to go to a homeopath. Consequently, we need to do the best we can with what exists. Nor does Ioannidis’ work mean that SBM is so hopelessly flawed that we might as well all throw up our hands and become reiki masters, which is what Freedman seems to be implying. SBM is our tool to bring the best existing care to our patients, and it is important that we know the limitations of this tool. Contrary to what CAM advocates claim, there currently is no better tool. If there were, and it could be demonstrated conclusively to be superior, I’d happily switch to using it.
To paraphrase Winston Churchill’s famous speech, many forms of medicine have been tried and will be tried in this world of sin and woe. No one, certainly not those of us at SBM, pretends that SBM is perfect or all-wise. Indeed, it has been said (mainly by me) that SBM is the worst form of medicine except all those other forms that have been tried from time to time. I add to this my own little challenge: Got a better system than SBM? Show me! Prove that it’s better! In the meantime, we should be grateful to John Ioannidis for exposing defects and problems with our system while at the same time expressing irritation at people like Freedman for overhyping them.

Should Evidence-Based Medicine Be Replaced by Interpersonal Medicine?
An opinion piece in the New England Journal of Medicine complains about the limitations of evidence-based medicine (EBM) and recommends a new approach they call “interpersonal medicine.” In my opinion, good clinical medicine is already interpersonal medicine; there is no need for something new.
Harriet Hall on November 27, 2018
Link

The 22 November 2018 issue of The New England Journal of Medicine carried an opinion piece by Stacey Chang, M.S. and Thomas H. Lee, M.D. titled “Beyond Evidence-Based Medicine.” They point out what they see as the limitations of EBM and recommend an improved approach that they call interpersonal medicine. In my opinion, they are misguided.
The limitations of EBM
In 1996, Sackett et al. defined EBM as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients”. That is an admirable and unobjectionable goal, but it has been distorted in practice. According to Chang and Lee, “guidelines based on clinical research are being hard-wired into our operational norms, incentive programs, and information systems”. The results of clinical research are being converted into a one-size-fits-all cookbook. That was never the intention.
Note the use of the words “judicious” and “individual patients”. Most published guidelines make it clear that they are not intended to be applied to every patient as a knee-jerk prescription. They are meant to guide physicians, not to dictate to them. The cholesterol guidelines emphasize that lifestyle modifications are crucial, and that “additional factors may be considered to inform treatment decision making”. The guidelines emphasize that prior to initiating statin therapy, the clinician and the patient should have a discussion covering:
     1.  The potential for ASCVD risk-reduction benefits
     2.  Potential for adverse effects and drug-drug-interactions
     3.  Heart-healthy lifestyle
     4.  Management of other risk factors
     5.  Patient preferences
     6.  Other factors such as family history
So obviously the limitations they describe are not inherent in EBM but are features of how the guidelines are mis-interpreted by some clinicians and regulators.
They say that some quality measures have already been retired because compliance with guidelines is uniformly high, but they contradict themselves by making the claim that “even when physicians prescribe medications that have been proved beneficial in randomized trials, the chances that patients are taking them a year later are akin, at best, to a coin toss.” They offer no supporting reference for that claim, nor do they offer evidence that stopping the medications has an adverse impact on clinical outcomes. There could be many reasons for stopping the medications, from side effects to cost and availability.
They say EBM shifted the center of gravity “away from the space between clinician and patient to somewhere between research and clinician.” I think that’s a straw man argument; but even if it is perceived that way by some clinicians, it was never the intention of EBM to cut the individual patient out of the loop.
Interpersonal medicine
They say that interpersonal medicine would recognize clinicians’ influence on patients, that it would be anchored in longitudinal, multidirectional communication, that it would broach social and behavioral factors, that it would require coordination of the care team, and that it would constantly evaluate its own approach.
They say that chronic diseases are rooted in social and behavioral determinants. They say obesity, diabetes, and heart disease “cannot be addressed effectively in isolated office visits.” They claim that meaningful interactions need to be focused on “motivation, engagement, empowerment, conviction, and resilience”, and that these are not limited to provider/patient encounters but may also occur between patients and families and communities. Through such interactions, relationships are built that can endure distraction and overcome hurdles”, they say:
Interpersonal medicine demands skills that place clinicians alongside their patient as fellow travelers and experienced guides…it also requires systems that draw on those skills when they’re likely to make a difference.
I would ask how they know when they’re likely to make a difference and what can guide their actions in treating an individual patient on the basis of social and behavioral factors if there is no evidence to guide them. They are advocating a return to the “intuition-based medicine” that was all we had before EBM and that was far less reliable as a guide to the truth.
They applaud the system at Boston’s Brigham and Women’s Hospital, where Medicaid patients are screened for social needs by practice assistants, alerting the clinician and also triggering actions by community health workers and others. But they don’t provide any evidence that that system improves outcomes, and they don’t explain why it should be limited to Medicaid patients.
They stress the importance of empathy, coordination, and communication, providing references to studies showing that physician communication is associated with a 19% gain in patients’ adherence to recommended therapies and improvement in a variety of outcomes. This is old news to most of us, and they don’t provide any guidance on how specifically to achieve meaningful improvement.
They recommend the development of matchmaking protocols to pair clinicians and patients for success, but they don’t provide any guidance as to how that might be done or any evidence that benefits would ensue. They praise clinics that have no waiting rooms: patients are taken directly to a room designed to accommodate conversation among patient, family members, and clinicians; no exam table is in sight; a chair converts to an exam table when necessary. That’s all very feel-good nice, but where is the evidence that it actually improves patient outcomes?
They point out that some solutions are already available but are inconsistently applied. There is a need for “building systematic capability at a scale that mirrors our scientific effort”. Finally, they conclude:
We can pursue an empathetic version of medicine that embraces emotion and appreciates behavior if we value human nature as much as human biology.
Good medicine is already interpersonal medicine
In my opinion, good clinical medicine already does all those things. There is no need for something new called interpersonal medicine. There is only a need for good clinical medicine to do what it already knows how to do well, even if it is not currently doing a good job of it. In my opinion, it would be injudicious to institute major changes in approach willy-nilly without first testing which changes produce meaningful results.
They are essentially advocating a return to “intuition-based medicine”, which they say wasn’t wrong but had limited access to meaningful data. I would argue that it quite often was wrong. The history of medicine is full of examples where clinicians were misled by intuition.
This is reminiscent of much of alternative medicine, integrative medicine, and functional medicine. They claim to treat the whole patient, stress prevention, and consider social and behavioral factors. But just as with this new proposal for “interpersonal medicine”, there is no evidence that they are actually accomplishing what they claim or that their approaches achieve superior results. In fact, there is evidence that patients seeing alternative providers are less likely to get the immunizations and screening tests that are based on good evidence.
Something missing
Their criticism of EBM is missing a key point, the one that led us to create the Science-Based Medicine blog.
EBM tends to focus on clinical trial results to the exclusion of scientific plausibility. The focus on trial results (which, in the EBM lexicon, is what is meant by “evidence”) has its utility, but fails to properly deal with medical modalities that lie outside the scientific paradigm, or for which the scientific plausibility ranges from very little to nonexistent.
This has led to unfortunate consequences, including quackademic medicine and the infiltration of non-science-based therapies and even outright quackery into our medical schools, clinics, and hospitals. It led the American Academy of Family Physicians to provide poor quality evidence about acupuncture, dry needling, and cupping, and to make the nonsensical practice recommendation that “electroacupuncture should be considered for patients with fibromyalgia.”
EBM is bad enough when legitimate research is misinterpreted and turned into cookbook mandates, but it’s far worse when it relies on Tooth Fairy science to support improbable treatments – especially when it unquestioningly accepts positive results of studies on something like homeopathy, which is incompatible with all we know about science.
Conclusion: Good medicine is already “interpersonal medicine”
The limitations they see in EBM are not inherent in EBM itself; they are a result of individuals misunderstanding and misinterpreting how EBM should be implemented. Rather than trying to go beyond EBM, it would make more sense to correct the abuses of EBM and go back to the original concept of judicious interpretation of the evidence, (with consideration of prior plausibility and compatibility with the rest of science) and applying evidence from studies not to mandate decisions, but to help guide decisions in the context of other unique factors affecting the individual patient.
The best clinicians already practice interpersonal medicine. Admittedly there are many things wrong with the way conventional medicine is currently being practiced. We could learn a thing or two from the way alternative practitioners please their patients (with charisma, compassion, touch, TLC, long appointment times, and the nonspecific provider/patient interactions that maximize placebo responses). We could do a better job of prevention and of helping patients change their lifestyles. But that doesn’t mean we need a new system of interpersonal medicine; it just means we can do better within the existing system. And as we make changes, we still need to rely on evidence to ensure that we really are doing better.

Science-based medicine isn’t just for CAM. The case of ivermectin shows that it never was.
Another large randomized controlled trial for ivermectin showed no efficacy for the early treatment of COVID-19. This is not a surprise to science-based medicine advocates. Here’s why the story of ivermectin shows that SBM isn’t just for “complementary and alternative medicine” (CAM) —and never was.
David Gorski on April 4, 2022
Link

A couple of weeks ago, I likened ivermectin to acupuncture. The reason why the comparison came to me is because the reaction of those promoting ivermectin as a highly effective treatment—miracle cure, even—for COVID-19 is the same approach to evidence demonstrated by advocates of “complementary and alternative medicine” (CAM)—or, as it’s now called, “integrative medicine” or “integrative health”—and especially by acupuncture advocates. Specifically, as more and more high-quality evidence from randomized clinical trials has failed to find a therapeutic effect from using ivermectin to treat COVID-19, increasingly its advocates point to positive studies that are less rigorous, such as observational and uncontrolled clinical studies. This is, more or less, exactly what acupuncture advocates have been doing as more and more high-quality studies with appropriate sham acupuncture placebo groups fail to find a detectable benefit for acupuncture for treating anything. They’ve been citing lower quality “pragmatic” studies, which might not be blinded (much less double-blinded), placebo-controlled, or, in some cases, even randomized. As I explain time and time again, though, citing pragmatic studies is putting the cart before the horse. Pragmatic studies are intended to see how well a treatment that’s been shown to work in high quality randomized controlled clinical trials works “out in the wild” outside of clinical trials and all the rigid protocols and selection criteria, a situation where the indications for the treatment inevitably expand as well. Ivermectin advocates even use the same sorts of excuses, too, when randomized controlled trials (RCTs) fail to show a benefit when ivermectin is used to treat COVID-19, such as claiming that medicine is biased and there is a double standard. (There is a double standard, but it doesn’t favor what acupuncture and ivermectin advocates think it does.)
In my post two weeks ago, I briefly mentioned an RCT of ivermectin for COVID-19 that was very much negative, but I didn’t discuss it extensively because it had not yet been published and had only been publicized in a news report from The Wall Street Journal. It turns out that last Thursday the study was finally published, in The New England Journal of Medicine, and, as described, it was a resoundingly negative trial, without even a hint of a whiff of efficacy. Was it a perfect trial? Of course not. No trial is. It was, however, large and well-designed and showed zero detectable effect from the early treatment of COVID-19 with ivermectin on hospitalizations and emergency room monitoring. As such, it was just one more drop in the drip-drip-drip of negative RCTs for ivermectin. Along with the drip-drip-drip of evidence that the most famous and largest RCTs of ivermectin for COVID-19 were either incompetently carried out or even fraudulent, and, as I mentioned, ivermectin for COVID-19 is looking increasingly like acupuncture for, well, anything.
So why bring this up again so soon after writing about it? I started thinking (always a dangerous thing) about science-based medicine (SBM) and the very purpose of this blog, and it occurred to me that the case of ivermectin is a very good real-world example of the utility of SBM and why evidence-based medicine (EBM) can fail for so long in a case like that of this particular drug.
Let’s go way, way back to the beginning, to explain the differences between EBM and SBM. Then I’ll explain how these differences apply. The primary difference is that SBM takes into account prior probability of a treatment working in evaluating clinical evidence.
SBM versus EBM
In the very first post ever on this blog all those years ago, SBM founder Dr. Steven Novella wrote:
All of science describes the same reality, and therefore it must (if it is functioning properly) all be mutually compatible. Collectively, science builds one cumulative model of the natural world. This means we can make rational judgments about what is likely to be true based upon what is already well established. This does not necessarily equate to rejecting new ideas out-of-hand, but rather to adjusting the threshold of evidence required to establish a new claim based upon the prior scientific plausibility of the new claim. Failure to do so leads to conclusions and recommendations that are not reliable, and therefore medical practices that are not reliably safe and effective.
This is why the authors of this blog strongly advocate for science-based medicine – the use of the best scientific evidence available, in the light of our cumulative scientific knowledge from all relevant disciplines, in evaluating health claims, practices, and products.
What did Steve mean by “prior scientific plausibility”? In brief, it’s an estimate of how likely a proposed treatment, when tested in an RCT or other clinical study, is likely to produce a positive result; i.e., a result consistent with the treatment “working” for the indication against which it is being tested. More specifically, it is an estimate of the prior probability of a given hypothesis before a study is conducted (in the case of an RCT that the null hypothesis will be rejected and there will be a statistically significant difference between the treatment group and the placebo group, indicating that the treatment works). I realize that this is boiling it down a bit, but I am writing for a lay audience.
It turns out that clinical trials are imperfect and can have a lot of “noise” even when designed and carried out perfectly. It also turns out that, from a Bayesian perspective, the prior plausibility that a treatment works matters a lot in interpreting clinical trial results; i.e., “posterior probability” the probability that a “positive result” is a “true positive” depends on the prior plausibility. (I’ll explain more in the next section what I mean by that.) Of course, in 2008, the original intent of SBM was to look at the evidence for treatments advocated as part of “alternative medicine”, CAM, or, as CAM is now more frequently called, “integrative medicine” or “integrative health”, both modalities in which “alternative medicine” (i.e., quackery) is “integrated” into conventional EBM. As I’ve argued, the newer names for CAM involving “integration” are nothing more than a rebranding of quackery.
Over the years, we’ve discussed many examples of alternative medicine with very low prior plausibility. Our favorite, as you might imagine, is homeopathy. The reason is simple. Based on homeopathy’s Law of Infinitesimals, which states states that the more you dilute a homeopathic remedy, the stronger it gets, we know that most homeopathic remedies are diluted to the point that it is unlikely that a single molecule of the original remedy is left. Homeopathy is thus arguably the purest example of a treatment with zero prior plausibility, given that most homeopathic remedies are is either water or another diluent. After all, as I like to say, for homeopathy to “work” for any disease or medical condition, several well-established laws of physics and chemistry would have to be not just wrong, but spectacularly wrong. It is, of course, true that there are other alternative medicine treatments that have a similar level of implausibility—impossibility, really—based on basic science considerations. (“Energy healing” comes to mind first.) However, homeopathy is so common and ubiquitous that it makes an excellent teaching example, which is why I use it so often. Also, I’ve noticed that a lot of people don’t even know what homeopathy is, including medical students and physicians. Many seem to think that homeopathy is just another form of herbal medicine. In any event, because many homeopathy remedies are just water used to make sugar pills, homeopathy is an excellent way to test the “noise” in clinical trials because it’s basically testing placebo vs. placebo.
So how does one consider prior plausibility in RCTs?
SBM vs. EBM: Prior probability/plausibility
We’ve long discussed the differences between EBM, which uses frequentist statistics, and SBM, which uses Bayesian reasoning that incorporates an estimate of prior plausibility
I’ve long complained about “methodolatry” in EBM, defined as the obscene worship of the RCT as the only valid method of investigation in medicine. It’s a term that I first learned in the context of countering misinformation about the H1N1 influenza vaccine back in 2009, taught to me by a senior epidemiologist. There’s a lot of methodolatry in EBM, and it’s part of the reason why treatments like acupuncture (whose prior plausibility, based on its pre-scientific concepts and mechanism of action is very, very low, albeit probably not zero) keep showing up as potentially working—or at least “needing more study”—to EBM practitioners. I would also like to take this opportunity right here to quote SBM co-founder and former regular Dr. Kimball Atwood, who discussed why EBM and SBM ought to be synonymous (but currently are not) and, more importantly why EBM is incomplete, and SBM is intended to complete it, or at least to fill in its “blind spot”. As Dr. Atwood put it, EBM “should not be without consideration of prior probability, laws of physics, or plain common sense” and “SBM and EBM should not only be mutually inclusive, they should be synonymous”.
Elsewhere, he argued:
That discussion made the point that EBM favors equivocal clinical trial data over basic science, even if the latter is both firmly established and refutes the clinical claim. It suggested that this failure in calculus is not an indictment of EBM’s originators, but rather was an understandable lapse on their part: it never occurred to them, even as recently as 1990, that EBM would soon be asked to judge contests pitting low powered, bias-prone clinical investigations and reviews against facts of nature elucidated by voluminous and rigorous experimentation. Thus although EBM correctly recognizes that basic science is an insufficient basis for determining the safety and effectiveness of a new medical treatment, it overlooks its necessary place in that exercise.
You can see where I’m going with this, I hope. In the post from which I drew that quote above, Dr. Atwood explained in more detail what I’m talking about in terms of Bayesian theory. It also discusses how an estimate of prior probability affects the posterior probability that a given p-value in a clinical trial indicates a “true” result, or, as Steven Goodman and Sander Greenland put it in 2007 put it, the “prior probability” of a hypothesis is its probability before the study, and the “posterior probability” is its probability after the study.
In fact, I think I’ll insert here a table that we at SBM like to cite to illustrate:

This is Table 2 from the 2007 article by Goodman and Greenland that illustrates how prior probability affects posterior probability for given p-values. Notice one thing. The lower the prior probability, the much lower the posterior probability for a given p-value, such that if a prior probability is estimated to be 1%, then the posterior probability of a result for a standard p-value less than or equal to 0.05 is only 14%, meaning that it’s only ~14% likely that the result is not a false positive under the conditions specified, which are commonly used conditions in designing RCTs. Even results with fairly low p-values bear serious questioning in the case of a treatment with a very low prior probability/plausibility. (For those of you who are more knowledgeable, the mathematical formulas and reasoning used to derive these numbers are in the reference.)
For something like homeopathy, in which the prior probability based on its scientific impossibility under currently understood science is zero, the situation is, of course, far worse than in the table above. I will also quote Steve Novella and myself from a 2014 paper that we coauthored, in order to address a common criticism of our argument that RCTs of highly improbable/implausible treatments (like homeopathy) are akin to testing whether magic works as medicine:
It should also be noted that ‘biologically plausible’ does not mean ‘knowing the exact mechanism’. What it does mean is that the mechanism should not be so scientifically implausible as to be reasonably considered impossible. In other words, the mechanism should not violate laws and theories in science that rest on far sturdier and longer established foundations than imperfect, bias-prone clinical trials. For example, homeopathy violates multiple laws of physics with its claims that dilution can make a homeopathic remedy stronger and that water can retain the ‘memory’ of substances with which it has been in contact before [9]. Thus, treatments like homeopathy should be dismissed as ineffective on basic scientific grounds alone. That is why we propose the term science-based medicine (SBM) as opposed to evidence-based medicine (EBM). SBM restores basic science considerations to EBM and is what EBM should be.
I’m assuming that most SBM readers accept that, for example, homeopathy or energy healing is so incredibly implausible/improbable from basic science considerations alone that basic science is all that is needed to reject it as a treatment. I’m also assuming that most SBM readers will accept that, for example, acupuncture, although not as improbable/implausible as homeopathy, is still incredibly implausible, with a prior probability reasonably estimated as very much less than 1%. (I’d say very much less than 0.0001%, even.) I’ll also emphasize right here that, while basic science alone can in the cases under discussion here be enough to reject a proposed therapy as so implausible as to be impossible and not worth testing in RCTs, basic science is never enough to demonstrate that a treatment works, no matter how plausible the mechanism and compelling the preclinical data in cell culture and animals. Many are the treatments that appeared highly promising before being tested in humans, only to fail.
But what about ivermectin for COVID-19? That doesn’t fall into the same categories as homeopathy or acupuncture, does it? It’s an actual drug that is highly effective against diseases caused by roundworm infestations. Its discoverers even won the Nobel Prize for that indication! Is it so implausible that ivermectin might also work against a viral illness like COVID-19? There was even a proposed mechanism for its antiviral activity against SARS-CoV-2, the coronavirus that causes COVID-19. That mechanism was even based on in vitro cell culture studies!
All of these are reasonable considerations. So join me as I discuss them in the last two sections of this post, in which I hope to convince you that the prior probability for ivermectin was always very, very low—not homeopathy-level low, admittedly, but quite low.
Ivermectin vs. SBM: Back to the future
So how did the idea come about that ivermectin might be an effective treatment for COVID-19 anyway? You might remember how the idea that repurposing hydroxychloroquine, an anti-malarial drug with immune modulating properties that make it useful as a mild immunosuppressive drug to use to treat autoimmune diseases, took hold early in the pandemic. In brief, Chinese physicians in Wuhan reported in early 2020 that none of a group of 80 patients with lupus erythematosus who were taking hydroxychloroquine went on to catch COVID-19. (Never mind that immunosuppressed patients were exactly the patients most likely to assiduously follow the recommendations of public health authorities during an epidemic.) A number of clinical trials were registered, and, based on anecdotal reports and small clinical trials (nearly all of which are as yet unpublished), in February the Chinese government published an expert consensus recommending CQ or HCQ for patients with COVID-19. Soon after, a number of nations followed suit. In addition, French scientist Didier Raoult started flogging hydroxychloroquine as a cure for COVID-19 and was soon joined by President Donald Trump and of course!—Dr. Mehmet Oz. The rest, unfortunately, is history. Even though by late summer 2020, it was becoming quite clear that hydroxychloroquine was ineffective against COVID-19, a conspiracy theory of a “suppressed cure” had been born.
The idea that ivermectin could be repurposed to treat COVID-19 is similar, but based on even less evidence. I discussed this evidence several months ago, when a conspiracy theory was being spread that Pfizer’s new antiviral drug Paxlovid was “Pfizermectin” because both ivermectin and Paxlovid have protease inhibition activity. I’ll just recap briefly.
It’s been known for years that ivermectin can inhibit coronavirus replication through the inhibition of a protein called α/β1 importin, something reported a decade ago. This particular protein is involved in the transport of proteins into the nucleus from the cytoplasm. In the case of SARS-CoV-2 infection, the transport of certain proteins into the nucleus is important for completion of its lifecycle. Now here’s the thing. Inhibition of this protein complex by ivermectin and preventing the replication of HIV and the Dengue virus requires fairly high concentrations (at least in terms of drug concentrations). The original Australian paper published in May 2020 that examined the ability of ivermectin to inhibit SARS-CoV-2 replication showed similar results, specifically the need for a very high concentration of drug to inhibit the importin that was needed for SARS-CoV-2 replication.
So what’s the problem? It’s basic pharmacology, as summarized in this review:
As noted, the activity of ivermectin in cell culture has not reproduced in mouse infection models against many of the viruses and has not been clinically proven either, in spite of ivermectin being available globally. This is likely related to the pharmacokinetics and therapeutic safety window for ivermectin. The blood levels of ivermectin at safe therapeutic doses are in the 20–80 ng/ml range [44], while the activity against SARS-CoV2 in cell culture is in the microgram range. Ivermectin is administered orally or topically. If safe formulations or analogs can be derived that can be administered to achieve therapeutic concentrations, ivermectin could be useful as a broad-spectrum antiviral agent.
Specifically, the IC50 (the concentration that produces 50% of maximal inhibition of a process) was in the 6 μM. Given that the molecular weight of ivermectin is 875 g/mol, 6 μM translates into ~5.25 mg/L or ~5.25 μg/ml, a concentration that is roughly 66-fold higher than the upper end of the range of blood levels of ivermectin safely achievable in a human’s bloodstream. Even this paper proposing ivermectin as a treatment for COVID-19 notes this serious problem:
A dose of 12 mg twice daily alone or in combination with other therapy for 5–7 days has been proposed as a safe therapeutic option for mild, moderate or severe cases of Covid-19 infection.10 The time to reach maximum plasma concentration of 20–50 ng/ml, after a dose of 6 or 12 mg, respectively is approximately 4 h.
This is almost certainly the reason that ivermectin doesn’t work against COVID-19 in spite of its activity in vitro against SARS-CoV-2. It requires a concentration roughly 66- to 197-fold higher than is safely achievable in the blood. That’s why the review concluded that maybe an ivermectin analogue that is either more active or can achieve a higher concentration safely in the bloodstream is worth investigating. Based on the proposed mechanism in the Australian paper, ivermectin was never a good candidate as a treatment for SARS-CoV-2. As I discussed for the claim that ivermectin should be considered a promising drug for COVID-19 based on its protease inhibitor activity, the situation is just as bad. Again, based on in vitro results, ivermectin was never a promising candidate as an antiviral drug to treat COVID-19.
But how do we translate this into prior probability? In the discussion of the NEJM trial to which I linked above, someone suggested what is probably an accurate assessment:
Yup. As liked to say, it wasn't homeopathy-level implausibility, but it was highly implausible.
— David Gorski, MD, PhD (@gorskon) March 31, 2022
I think that a pretest probability of much less then 1% is accurate given in vitro results like the experiments that I described. I also noted on Thursday after seeing the NEJM study:
To accept that such a mechanism might be operative, the clinical trial results have to be highly compelling and pristine. None of the clinical trials for #ivermectin even came close to that standard.
— David Gorski, MD, PhD (@gorskon) March 31, 2022
In other words, if ivermectin were to be actually useful against COVID-19, it would almost certainly have to work by a different molecular mechanism than the one described in the in vitro study cited because it’s known to be impossible to safely achieve ivermectin concentrations in human blood that are sufficient even to inhibit viral replication by 50%—or even anywhere near such concentrations. Of course, it’s always possible that such a previously unknown mechanism was operative, but highly unlikely. Moreover, to consider that such a mechanism might be operative, the clinical trial results would have to be pristine in terms of very compelling results from very well-designed and executed clinical trials, or, as Steve Novella once put it, “When the basic science dictates that a proposed treatment is highly implausible, the bar for clinical evidence should be raised proportionately”.
This never applied to any clinical trial of ivermectin for COVID-19, not even the seemingly strongly “positive” ones, and that’s even leaving aside the discovered incompetence and likely fraud in the largest “positive” trials, which led to falsely “positive” meta-analyses.
As I said, these negative results never surprised those of us with SBM tendencies. Given the very low prior probability that ivermectin is effective against COVID-19 based on in vitro data, even the best “positive” clinical trials likely had a low posterior probability. Moreover, they weren’t the only trials. Most trials were equivocal or negative, and, as is the case with acupuncture, the larger and better designed the study, the more likely it was to be negative, which leads me to an adage that I frequently use on Twitter (because it’s pithy) but have not yet (as far as I can recall) repeated here on SBM:
Very low prior plausibility
+
Equivocal clinical studies
=
Drug doesn’t work for the proposed indication
Or at least, any effect observed will be too small and inconsistent to be clinically useful.
SBM: It’s not just for CAM anymore. (It never was.)
It’s been a long time since we’ve discussed the difference between EBM and SBM in quite this much detail, discussions like this having been common here around 2008-2010, but I felt that a discussion like this was overdue. The reason, of course, has been the pandemic and how EBM has treated highly implausible COVID-19 therapies in the same way that it’s long treated highly implausible CAM therapies, through the lens of methodolatry, in which only RCTs matter for determining if a treatment does or doesn’t work and in which even flawed RCTs trump basic science—except that they don’t, at least when Bayesian considerations are used.
What makes the discussion of ivermectin interesting to me in this context is that it is an example of why SBM matters in all areas of medicine, not just the consideration of “integrative” quackery. SBM mavens all immediately realized that, even if you accepted the rationale proposed at face value, ivermectin was incredibly implausible as an effective therapy for COVID-19 just based on the in vitro data alone. It’s basic pharmacology. A drug that only inhibits the target protein at a concentration that is at least nearly 70-fold higher than the highest blood concentration of drug that can be safely achieved using standard dosing is incredibly unlikely to be an effective treatment. It’s also a general principle that most highly effective drugs inhibit their target at nanomolar or ng/ml concentrations, not micromolar or μg/ml concentrations. Candidate drugs that only inhibit their target at such high concentrations, in general, tend to be incredibly unlikely to be useful drugs.
Indeed, any pharmaceutical company that developed a drug just like ivermectin for COVID-19 would have abandoned it after in vitro testing showing that it required such a high concentration to inhibit the intended target. A drug company would have deemed such a candidate compound as not worth pursuing further, except maybe as a base molecule to chemically modify so that it either inhibits the desired target at a much lower concentration or becomes able to achieve much higher blood levels safely. Yet a number of scientists whom I respect were, until very recently, saying that ivermectin “probably doesn’t work” or even saying that there was a strong developing evidence base; that is, until it all fell apart with the determinations that a couple of the largest, most “positive” studies couldn’t possibly have been carried out as reported. Basic pharmacology matters.
I also realize that ivermectin believers won’t accept an SBM-based argument against ivermectin any more than believers in acupuncture or homeopathy accept SBM-based Bayesian arguments against their favorite woo. Indeed, in reaction to the NEJM study, there were a lot of reactions like this:
Effect of Early Treatment with Ivermectin among Patients with Covid-19 | NEJM, FAKE NEWS https://t.co/fMxY0LYuQN
— skp (@pt34tide) April 1, 2022
And this:
All of them fell under the sorts of “rationales” as in this meme, which I most definitely am stealing:
I'm stealing that meme. pic.twitter.com/dthJgAAKR8
— David Gorski, MD, PhD (@gorskon) March 31, 2022
However, remember that EBM arguments don’t sway believers in homeopathy or acupuncture, either; so one should not expect that EBM or SBM arguments would sway ivermectin believers. My hope is that SBM thinking will be more likely to sway EBM adherents who don’t really take into account prior plausibility in evaluating RCT evidence and therefore take much longer to reach a conclusion that is obvious to SBM about a treatment, in the case of ivermectin that it doesn’t work against COVID-19. Indeed, at this stage I would argue that it is unethical to begin another RCT of ivermectin to treat COVID-19. Given how resoundingly negative the evidence is now, such a trial would be all risk with no realistic potential for any patient to benefit. I’m not even sure whether it’s ethical to continue to recruit patients to RCTs of ivermectin still enrolling.
In future talks about the differences between SBM and EBM and how EBM should be (but still is not) synonymous with SBM, I plan on adding the example of ivermectin for COVID-19 to my long-used examples of homeopathy and acupuncture when I discuss Bayesian thinking and prior probability/plausibility because SBM isn’t just for CAM any more. It never was, and never should have been. If I’ve been guilty of not applying it to conventional medicine as much as I do to CAM, the case of ivermectin has shown me the error of my ways.

   CAM and Evidenced-Based Medicine
Steven Novella
Link
Mark Tonelli, MD has problems with evidence-based medicine (EBM). He has published a few articles detailing his issues, and he makes some legitimate points. We at science-based medicine (SBM) have a few issues with the execution of EBM as well, so I am sympathetic to constructive criticism.
In an article titled: Integrating evidence into clinical practice: an alternative to evidence-based approaches. The abstract states:
Evidence-based medicine (EBM) has thus far failed to adequately account for the appropriate incorporation of other potential warrants for medical decision making into clinical practice. In particular, EBM has struggled with the value and integration of other kinds of medical knowledge, such as those derived from clinical experience or based on pathophysiologic rationale. The general priority given to empirical evidence derived from clinical research in all EBM approaches is not epistemically tenable. A casuistic alternative to EBM approaches recognizes that five distinct topics, 1) empirical evidence, 2) experiential evidence, 3) pathophysiologic rationale, 4) patient goals and values, and 5) system features are potentially relevant to any clinical decision. No single topic has a general priority over any other and the relative importance of a topic will depend upon the circumstances of the particular case. The skilled clinician must weigh these potentially conflicting evidentiary and non-evidentiary warrants for action, employing both practical and theoretical reasoning, in order to arrive at the best choice for an individual patient.
I certainly agree that clinical evidence (what he he referring to by “empirical” evidence above) is not, and should not be, the sole type of knowledge that is incorporated into clinical decision-making. However, I think this criticism is a bit of a straw man, at least with regard to items 2, 4, and 5. The goals and values of the patient are definitely part of clinical decision-making, even in a rigorously evidence-based practice. We are, after all, treating people, not diseases. When I was in medical school this was called the biopsychosocial model of medicine. Now it is also not uncommon for quality of life measures and overall satisfaction to be incorporated as outcome measures in clinical trials, blurring the lines between empiricism and personal goals and values.
So while I agree that patient values and goals absolutely need to be taken into consideration when practicing medicine, I don’t see this as a new idea or one that is at odds with EBM, nor entirely distinct from empiricism. By including this as he does, however, there is the implication that EBM excludes such considerations, and I do not believe that is fair.
Where we likely mostly agree is on number 3 – pathophysiological rationale. I could expand this to include all of basic science – medical practices should be plausible. I also think he has a legitimate point in that EBM gives too much emphasis to clinical science and shortchanges basic science. But it is interesting to note that the EBM grading system for recommendations do allow for extrapolation (i.e grade B=.consistent level 2 or 3 studies or extrapolations from level 1 studies). Extrapolation involves considering pathophysiology and mechanism of action. While extrapolation (rather than direct evidence) downgrades the recommendation by one category (which is appropriate) it does not exclude it altogether.
Further, I think the real problem with failing to consider pathophysiology is not for support of a plausible treatment, but to be extra cautious about an implausible treatment. When the basic science dictates that a proposed treatment is highly implausible, the bar for clinical evidence should be raised proportionately. I don’t think this is what Tonelli had in mind, however, as we will see.
Item #2- Experiential evidence, is highly problematic. While experience is great for some things, like recognizing diagnoses, being sensitive to the subtleties of history taking, and interfacing with patients – it is highly misleading when it comes to determining safety and efficacy. The simple fact is that personal experience is too limited, quirky, and uncontrolled, and is overwhelmingly likely to simply confirm our biases than actually lead us in the direction of truth.
In another related article (actually published in 2001, earlier than the 2006 paper above), Tonelli clarifies:
Empirical evidence, when it exists, is viewed as the “best” evidence on which to make a clinical decision, superseding clinical experience and physiologic rationale. But these latter forms of medical knowledge differ in kind, not degree, from empirical evidence and do not belong on a graded hierarchy.
He is partly correct here – these other forms of evidence are not necessarily below, but are tangential to, empirical evidence. But I think Tonelli is missing the context of EBM. EBM is not a method for solely determining clinical practice (clinical decision-making) but for determining safety and efficacy, which is one factor that informs practice. Values, the system, and the human side of medicine also go into clinical practice, but they should not be used to determine efficacy. So it seems his criticism is based upon a straw man constructed of his own confusion.
I might have been inclined to give Tonelli some benefit of the doubt, were it not for this:
The methods for obtaining knowledge in a healing art must be coherent with that art’s underlying understanding and theory of illness. Thus, the method of EBM and the knowledge gained from population-based studies may not be the best way to assess certain CAM practices, which view illness and healing within the context of a particular individual only. In addition, many alternative approaches center on the notion of non-measurable but perceptible aspects of illness and health (e.g., Qi) that preclude study within the current framework of controlled clinical trials. Still, the methods of developing knowledge within CAM currently have limitations and are subject to bias and varied interpretation. CAM must develop and defend a rational and coherent method for assessing causality and efficacy, though not necessarily one based on the results of controlled clinical trials. Orthodox medicine should consider abandoning demands that CAM become evidence-based, at least as “evidence” is currently narrowly defined, but insist instead upon a more complete and coherent description and defense of the alternative epistemic methods and tools of these disciplines.
This casts a new light on all of Tonelli’s other publications. It seems he is making an elaborate argument for the inclusion of other kinds of evidence (other than rigorous, controlled, clinical studies) as support for fanciful but ideologically appealing treatments.
This is a refrain that is becoming common in the CAM community - that we need to redefine “evidence”, not restrict ourselves to narrow definitions of evidence, and that CAM modalities cannot be properly studied by traditional scientific methods. There is always a flavor that CAM must free itself from the tyranny of scientific evidence.
What is it, exactly, about scientific methods that they feel is incompatible with CAM methods – being thorough, counting all the data, controlling for variables, minimizing the effects of bias, carefully defining terms and outcomes, or being statistically rigorous? Even individualized treatments can be studied rigorously – so that is an insufficient excuse. In the end, the call to expand the definition of evidence is just a deceptive way of asking for sloppy methods of research, because CAM modalities generally do not hold up under rigorous standards.
We don’t need to redefine or expand the methods of science – we need to return common sense to medicine.

Has science-based medicine already lost to pseudoscience?
David Gorski on March 10, 2014
Link

After writing Saturday’s 5,000-word magnum opus about misguided “right to try” bills that are proliferating in state legislatures like so much kudzu, I thought I’d try something a bit different—and more concise. Fear not. This doesn’t mean that I’m going to become Harriet Hall as a writer, because no one does concise and insightful as well as she does, but I do on occasion want to try my hand being less logorrheic. I’ll probably fail, but at least I can pat myself on the back for trying. If I succeed, though, it’ll only make me better. I hope. I also realize that I just made it harder by blathering on for a whole paragraph before getting to the point, a habit of mine that infuriates some readers and amuses others who find my way of winding towards the point at least somewhat entertaining.
Thus endeth the nauseatingly—but briefer than usual!—self-deprecating navel gazing and beginneth the post.
The opportunity appeared to me in the form of an article that popped up in my feed on Medscape entitled, Do Clinicians Base CAM Recommendations to Patients on Evidence of Efficacy? Since “complementary and alternative medicine” (CAM) is, by and large, mostly made up of a collection of modalities either based on prescientific thinking and possessing little, if any, plausibility on a scientific basis, my first reaction was to note that health care practitioners do recommend CAM to some patients, meaning that the answer must be, “No,” and then to move on. However, I wanted to see what Dr. Désirée A. Lie, the author, said and to see what the reasons are for whatever answer she came up with. So I read on.
The article starts with a case study:
A 50-year-old man with a body mass index in the normal range presents to your office stating that he would like to be more active in outdoor sports, but he is unable to participate because of chronic low back pain. Vertebral disc prolapse had been excluded by earlier imaging, which was notable for mild osteoarthritis of the lumbar spine. In the past, physical therapy has been of limited help, although he continues to do back stretches daily. He spends 2-4 hours commuting by car to work each day and runs or walks 2-3 times a week, 30-40 minutes at a time.
The patient is asking for your recommendation for a complementary or alternative medicine (CAM) option that would improve his functioning and ability to participate in more vigorous physical activity. He does not wish to take anti-inflammatory agents because of reflux esophagitis.
Notice how the question is framed. The patient is asking for CAM. That puts a physician in a tough spot. The patient wants something magical to make his low back pain go away, and, unfortunately, for low back pain there is nothing magical. My first recommendation would be that the man find a way not to have to drive so much, because if there’s one thing that’s hard on a person with low back pain, it’s driving for hours every day in a car. If that’s impossible, I’d ask the man if he could find a way to have better lumbar support and plenty of room. Or, I might suggest a few things but then tell the patient what he doesn’t want to hear, namely that if those interventions don’t work his desire to be more active in outdoor sports might not be feasible. Be that as it may, though, many docs would probably recommend some sort of “CAM” to this patient, although I would note that recommending more or different exercises or other lifestyle modalities represent what should be science-based medicine, not CAM. CAM has appropriated many of these modalities, such as lifestyle interventions or nutrition, as somehow being “alternative” when they are not.
Be that as it may, there are two questions in a survey form:
     1.  Which of the following healthcare professionals is likely to recommend CAM for the patient’s low back pain?
     2.  What modalities are healthcare professionals most likely to recommend?
The choices for the first question included physicians, nurses, and pharmacists—and, of course, “all of the above.” The choices for the second question included osteopathic or chiropractic manipulation, acupuncture, massage, and, again, “all of the above.” Disappointingly, by far the most common answer given for the second question was “all of the above” (61%), while for the first question the most common answer was “a nurse” (45%), followed by “they are all likely to recommend CAM” (40%). “All of the above” was listed as the “correct answer” for both questions, based on this explanation:
Since the 1990s, national survey data in the United States have reported a prevalence of CAM use that has risen from 1 in 3 patients to nearly one half of all patients.[1-4] A recent review of the 2007 National Health Interview Survey examined self-reported CAM use by healthcare workers (categorized as providers, such as physicians and nurses; technicians, such as sonographers and laboratory technicians; and support workers, such as nursing aides) using the National Center for Complementary and Alternative Medicine taxonomy (alternative medical systems, biologically based therapies, manipulative body therapies, mind/body therapies, and energy-healing therapies).[5]
The authors found a higher prevalence of use in the past year among all healthcare workers compared with the general population (41% vs 30%). As well, 76% of all healthcare workers reported using at least 1 modality in the prior year. Those working in the ambulatory setting were more likely than hospital workers to use CAM. Healthcare providers (such as physicians, nurses, and pharmacists) had a 2.2 times increased odds of seeking practitioner-based CAM modalities (such as acupuncture and manipulation) and 2.7 times increased odds of self-treated CAM modalities (such as botanicals or supplements) compared with support workers.
The most common reason given for CAM use by healthcare workers was “general wellness,” and the most commonly treated condition was anxiety in this study. There was also a significant increase since 2002 in use of the following modalities by healthcare workers: acupuncture, deep breathing exercise, massage therapy, meditation, naturopathy, and yoga.
I can’t help but note that the very first sentence promulgates a common myth about CAM, namely that anywhere from one-third to one-half of patients use it; i.e, that it’s so popular that it should be considered mainstream. As bloggers here at SBM, such as Steve Novella, Brennen MacKenzie, and myself, have explained, this is indeed an exaggeration, but it’s a useful exaggeration, given how it feeds CAM media myths. As Steve Novella has pointed out, for instance, that, contrary to a picture of increasing CAM use, CAM use for all categories except massage are all either static or only slightly increased. More importantly, as all of us have been pointing out again and again, these CAM numbers are inflated by including items that shouldn’t necessarily be considered outside of mainstream medicine, such as massage, biofeedback, and yoga—which, let’s not forget, is merely exercise and stretching. It’s only when pseudoscientific claims are made (as is not infrequently the case for nutrition and exercise modalities like Tai Chi and yoga) that these modalities fall outside of the mainstream and become “alternative.” Also, manipulative therapy is included. As I like to say, I have little problem with chiropractic as long as chiropractors use only manipulative therapies similar to those used by physical therapists, which is science-based medicine. I only have a problem with chiropractors when they go beyond that—which, unfortunately, all too many of them do—and become what I like to call physical therapists with delusions of grandeur, claiming to be able to treat allergies, asthma, and all manner of disease not based on the spine or the musculoskeletal system. In any case, chiropractic and osteopathic manipulation together add up to around 21%, and yoga adds nearly 10% to that.
Yes, Dr. Lie fell into that trap. She also cites a study published in 2012 claiming that CAM use is more prevalent in health care workers and providers (41% compared to 30% in the last year). If you look at the study in a bit more detail, though, you’ll soon find that it has the same issues as the 2007 NHIS had, and if you look at the numbers for individual modalities (which you can, as the study’s available at PubMed Commons for free), particularly in Table 2, you’ll see that, once again, they are dominated by manipulative therapies, 21.7% reporting having used chiropractic or osteopathic manipulation. If you look at biologically based therapies, which includes any diet-based therapy, such as vegetarian diet, macrobiotic diet, Atkins diet, Pritikin diet, Ornish diet, Zone diet, South Beach diet, as well as any use of dietary supplements, you’ll notice that 68.9% report some form of self-treatment, but only 0.3% doing “practitioner-based” therapy. One wonders whether anyone who put himself on a diet to lose weight without having a practitioner supervise it would count as having used CAM by this definition. One notes a similar issue with “mind-body” therapies. While 30.8% of health care practitioners reported using them as “self-treatment,” only 0.9% reported practitioner-based use of these therapies. One notes that yoga and Tai Chi are listed as falling under this category (supplemental information); so it’s not at all surprising that one in three people might have tried yoga or Tai Chi at some time in their lives. My wife’s done yoga. My uncle used to do yoga before it was cool to do it. I’ve thought of trying it myself to get into better shape. It’s exercise, people!
The rest of the modalities listed in Table 2 show single-digit percentages for everything else, such as energy therapies, which would include reiki, therapeutic touch, and various other forms of “energy healing” (1.5%), and alternative medical systems, which would include homeopathy and ayurveda (3.0% self-treatment, 2.6% practitioner-based treatment). Next, if you look at the odds ratios in Table 4, you’ll see that, disappointingly, health care providers have an odds ratio of having used CAM in the last year of 2.6 (95% confidence interval 1.7-4.2), leading the authors to conclude:
Even with these limitations, our results are suggestive of why CAM therapies are increasingly integrated into health care. There is evidence that personal use of CAM by health care workers is related to the provision of, referral for, or general openness to the integration of CAM therapies in health care practices. For example, Tracy et al. (2005) reported a strong correlation between personal use of specific CAM therapies among critical care nurses and the use of those same CAM therapies in practice. Thus, personal use of CAM by health care workers may be a principal determinant in the movement toward “integrative care”—the mainstreaming of CAM with allopathic medicine (Mann, Gaylord, and Norton 2004; Winnick 2005). In addition, in the context of recent federal health reform changes, in 2014 when the health insurance exchanges begin, states may be more ready to license practitioners of various CAM therapies and thus require insurance coverage for CAM.
In other words, more health care providers are using CAM, at least if you believe this study and Dr. Lie’s article. The implication (and apparent hope) among CAM advocates is that this means that CAM will increasingly become more tightly “integrated” with real medicine. True, we are left with the proverbial “chicken or egg” problem in that it’s not clear whether the reason health care practitioners are apparently using more CAM, however defined, is because of the propaganda promoting its use or whether CAM is becoming more “integrated” with medicine because more and more physicians are embracing it. It could well be a vicious cycle, in which increasing “mainstreaming” of CAM through its infiltration into academic medical centers and the medical school curriculum feeds greater acceptance among physicians and therefore greater usage. While it is true, if Dr. Lie’s article is correct, that CAM use is highest among nurses, followed by pharmacists, and then by physicians, it is noted that although the average of reported usage by physicians is the lowest, geographic variability in usage among physicians is the highest, ranging from only 24% of physicians in Denver having reported ever using CAM (I really didn’t see that one coming, given Colorado’s well-known predilection for embracing quackery) to 49% of physicians in Kentucky and up to 83% of primary care physicians at a medical school (it was the Morehouse School of Medicine, for anyone who’s interested).
It all makes me wonder whether medicine has reached a tipping point, if the CAM genie is out of the bottle and can’t be put back in. A while back, I wrote about how Andrew Weil was spearheading a plan for a board certification in “integrative medicine,” which I dubbed the “ultimate triumph of quackery,” because of how it was the next step in the legitimization of “integrating” quackery with scientific medicine. Even though CAM practitioners were very cool to the idea because of turf issues (namely, only physicians could have a board certification in “integrative medicine”), physicians are apparently flocking to this new certification, so much so that the first ever integrative medicine board exam has been postponed, as announced on the American Board of Physician Specialties website:
Since announced in mid-2013, interest in the new American Board of Integrative Medicine (ABOIM) has been overwhelmingly positive. An unprecedented number of applications were received by the December 1st deadline for the first exam administration, originally scheduled for May 2014, and this tremendous demand affected our projected timelines. In an effort to ensure the highest level of customer service to both current and future applicants, we have postponed the first administration of the examination until November 2014.
It’s times like these that I wonder whether the forces of pseudoscience have already won. But then I reassure myself by pointing out that, if you exclude exercise, diet, and manipulative therapies, vanishingly small numbers of physicians and other health care providers use anything that could in any way be considered “CAM.” Most physicians don’t use CAM themselves or prescribe it for their patients. From my anecdotal experience, most physicians appear dismissive of CAM. Unfortunately, they also don’t stand up for science-based medicine (or even evidence-based medicine) enough, either. Rather they are “shruggies,” whom Val Jones defined as “fairly inert,” unwilling to “argue the merits (or lack thereof) of complementary and alternative medicine (CAM) or pseudoscience in general,” because they just “aren’t all that interested in the discussion, and are somewhat puzzled by those who are.” That leaves it to us, the ones who are interested—such as the Society for Science-Based Medicine, which those of our readers who haven’t joined yet should totally join now (not so subtle hint, hint)—to sway them and hold our fingers in the dike protecting SBM from the ever-pummeling waves of quackademic medicine that have been pounding against it for the last 30 years ago. Stories like these remind me that we need to do more than just hold our fingers in the dike.

2020 – The Good and the Bad
What 2020 has taught us about the power of science and the harm of pseudoscience.
Steven Novella on December 23, 2020
Link

There is no doubt that 2020 will be remembered as an extraordinary year, mostly because of the worst pandemic in a century. Worldwide the pandemic has infected more than 78 million people with more than 1.7 million deaths. In the US there have been more than 18 million infections and more than 321,000 deaths, which is almost certainly an underestimate. As the year comes to a close the pandemic is far from over – we are in the middle of the third and largest wave, which will likely get a boost from holiday gatherings, and with a long winter yet to come.
The pandemic has exposed the best and worst of humanity, and highlights just about everything that we deal with at Science-Based Medicine. Let me start with the good news. The emergence of a novel coronavirus, SARS-CoV-2, which causes COVID-19, was partly a scientific challenge to the world. At the end of 2019 there was an outbreak of an infectious respiratory illness in Wuhan, China, with the first 4 cases being identified on December 29th. By January 9th the WHO announced the outbreak of a possible coronavirus illness in the region, beginning travel advisories. By January 12th the virus itself was isolated from epithelial cells taken from infected patients. On January 24, the Chinese scientists who identified the virus published a complete genomic sequence in the NEJM.
In less than a month from the first warning that a respiratory illness of unknown origin was possibly identified in a hospital in Wuhan, we had the complete genetic sequence of the responsible virus. Amazingly, with the genetic sequence in hand, it took Moderna just 2 days to develop a vaccine. It may be frustrating to know that we have had an effective vaccine against the pandemic the entire time, but we only know that in hindsight. The months were spent testing the vaccine for safety and efficacy, in an expedited process, but still it takes time. Meanwhile many other vaccines were in development as well. By the end of 2020 we now have more than 50 vaccines in development around the world, with several at least preliminarily approved. In the US the Moderna vaccine and another mRNA vaccine from Pfizer-BioNTech have EUA’s and are being distributed, with more than 2 million doses already given.
We do have to marvel at that timeline – in less than a year we went from an unknown novel infectious illness to multiple effective vaccines being distributed. In the future that timeline could be further compressed. Once the mRNA vaccine platform is proven safe and effective, the clinical trial phase can be further streamlined. Again – the vaccine itself existed within two days, we just need to decide how much testing is necessary prior to distribution.
During this time we also learned a great deal about the virus and the disease it causes. Within the first six months there were over 23 thousand scientific papers published about the pandemic. We learned about the typical course of the disease, best practice for treatment, vulnerable populations, and developed some therapeutics. The illness case fatality rate has been decreasing due to this improving knowledge. We also learned how the virus spreads and how to prevent it. I think by any measure we have to consider the scientific response to the challenge of this pandemic a stunning success.
We can also consider the professional response – how did the medical community handle the challenge of being swamped with cases of a new and serious illness? There, too, I think the robustness and professionalism of the medical community came through. Healthcare workers calmly risked their lives to treat patients. Hospitals rapidly developed new protocols to minimize spread. Intensive care units had to make do at times with insufficient resources. Overall, it was a heroic effort that saved numerous lives and helped contain this pandemic. In one estimate, by September in the US more than 1,700 healthcare workers died from COVID-19.
From a political perspective, the response of governments to the challenge of the pandemic was mixed. Some countries did well, while others failed. The US, unfortunately, did worse that most comparable nations, with a too-little-too-late response in almost every area. We had too little PPE (personal protection equipment) which contributed to the healthcare worker and other frontline worker fatalities. We had too little testing and contact tracing. We lacked a coherent national strategy, or clear mandates coming from the top that the public should wear masks and socially distance, with blatant public violations by some leaders. We wasted a shutdown which only delayed the spread, by failing to combine it with adequate testing and therefore were not able to calibrate the reopening optimally.
The COVID pandemic also provoked a parallel pandemic of misinformation, conspiracy theories, and pseudoscience. This was entirely predictable, especially by those engaged with science communication focused on countering pseudoscience. Of course, magic CAM treatments came out of the woodwork. The anti-vaxxers started opposing COVID vaccines before they even existed. And there were the pandemic deniers – denying that masks work, downplaying the severity of COVID, or denying the pandemic altogether.
In all, the pandemic served as a giant example of “what’s the harm” when it comes to medical pseudoscience. Allowing conspiracy theories, lack of trust in experts and professionals, anti-science attitudes, and pseudoscience to simmer in a society has demonstrable harm. One such harm is that it weakens our ability to respond to a challenge like the pandemic. It’s impossible to calculate how many excess deaths there have been as a result, but it is plausible (looking at other countries with a more optimal response) that the US could have kept pandemic deaths to less than 100,000 with a more coordinated response. This means that (again, we cannot prove what would have happened, but it is plausible) over 200,000 excess deaths occurred (so far) because of our bungled response, which was due to a failure to properly listen to the experts and follow their advice.
The pandemic highlighted both the growing power of our advancing science, and the harm of rejecting science and expertise for conspiracy theories and pseudoscience. I doubt we will collectively learn this lesson, but at least we will do our best to point it out whenever we can.

After 15 years of SBM: Lessons learned and what the future holds
Last week, Dr. Novella discussed what SBM has accomplished over the last 15 years. I’m going to discuss lessons learned, what has changed, and remaining huge challenges. Unfortunately, after the pandemic, our position in 2022 reminds me even more than ever of Aragorn at the Black Gate of Mordor, but that does not mean things are hopeless.
David Gorski on December 26, 2022
Link

It shocked me to realize this weekend as I sat down with my laptop to compose this post that it will (probably) be my last SBM post of 2022. Dr. Steve Novella’s post last week also reminded me that this blog is fast approaching its 15-year anniversary. That’s right. On January 1, 2008, Steve introduced the blog with a post entitled, simply enough, Announcing the Science-Based Medicine Blog. He then followed up the next day with a post about the false dichotomy between plant-based medicine and pharmaceuticals to address a common trope that somehow “natural” (i.e., plant-based) medicine is better than purified pharmaceuticals. Next up, Dr. Kimball Atwood, now retired from practice and blogging, introduced concepts with which our readers have become familiar, namely how randomized controlled clinical trials (RCTs) of homeopathy reveal the shortcomings of evidence-based medicine (EBM). Indeed, it is an example that we’ve also come back to again and again of modality that is totally implausible if not outright impossible from a basic science standpoint to undergird our our central argument that prior plausibility rooted in basic science and prior scientific evidence must inform clinical data because what happens when that isn’t the case and “pure” EBM is used is “complementary and alternative medicine” (CAM), which was rebranded as “integrative medicine,” and, more recently, “integrative health.” What happens is quackademic medicine, in which ostensibly science-based academic institutions like the Cleveland Clinic and UCSF embrace “integrating” quackery like acupuncture, naturopathy, and homeopathy into medicine and, as Steve put it, the exploitation of EBM to promote a worthless treatment based on unscientific notions. Such were the main concerns of this blog in the early years.
Although I cringed a bit when I reread the post, I first introduced myself on this blog, I chose to riff on a memorable quote from Admiral James Stockdale, Ross Perot’s running mate who famously introduced himself during the first 1992 Vice Presidential debate with, “Who am I? Why am I here?” The reason was that, at the time, I was largely unknown under my real name, although I had built a blog following on my not-so-secret other blog. Fifteen years on, I’m in some ways not much better known and in others too well known, having, like nearly everyone on SBM, endured intermittent attacks from quacks and antivaxxers, including threats of lawsuits and even a couple of lawsuits, one against Steve and one against several editors, including retired ones. Truth be told, I still sometimes feel like that unknown pseudonymous blogger who was invited 15 years ago to join this enterprise. But I’m not. Not anymore. Not after 15 years. After 15 years of dealing with medical misinformation, I sometimes feel like Aragorn at the Black Gate of Mordor in The Lord of the Rings. But I continue, and I hope to rouse the troops, metaphorically.

Now I’m ostensibly the managing editor of SBM. (I say “ostensibly” because in practice I rarely do much actual editing, at least not of any of the regular bloggers (particularly the remaining founders of SBM, such as Drs. Harriet Hall and Steve Novella) and now primarily restrict myself to choosing and editing guest posts. More on that later.) In that role, I’d be lying if I were to claim that the landscape with respect to science and medicine has improved since we first started this quixotic effort in 2008. Now is as good a time as any for me to look back and forward. Steve concentrated mainly on what SBM is, what we’ve accomplished, and what’s next. I’m going to contrast more on lessons learned, particularly from the pandemic, what I view as the single biggest challenge to SBM as a concept and a blog now, and some ideas on what to do next.
Successes in context
In his post last week, Steve outlined some successes of SBM, and I have to agree. First, there’s the sheer volume of what we have produced, with a new post (nearly) every weekday (and some weekends) for 15 years:
…we have published 4,328 articles covering just about every topic in the science of medicine. Hopefully we have at least added a useful voice to the much needed conversation surrounding how best to apply the latest science to the practice of medicine.
Indeed, I like to think of this as perhaps our greatest ongoing contribution and utility. Because a number of our posts have ranked high in Google searches of various topics, such as homeopathy and various antivaccine claims (the latter particularly true since the pandemic hit), SBM is now well-established and has become a resource for journalists looking for a science-based viewpoint for stories about various medical issues. So have our bloggers, whose commentary is featured in various news stories. Before SBM, I had never been contacted by a reporter to comment on a medical story. Now, while I wouldn’t say it’s common I can say that it’s not uncommon. Generally several times a year I’ll be asked to comment, and sometimes my pearls of wisdom even show up extensively in stories.
Still, I also have to agree with Steve when he wrote:
It is, of course, very difficult to assess the impact of SBM. There is no way to know what the world would have been like had we not engaged in this project. There is no controlled experiment. We must make do with anecdotal evidence. We frequently hear from practitioners who find our articles and our perspective useful in their practice, even changing how they practice medicine. We are often a resource for the media who would otherwise have only promotional or gullible voices on topics involving pseudoscience in medicine. And we hear from many health care consumers who were able to make a much more informed decision before turning to “alternative” interventions. I like to think we even influenced the FDA and FTC when they were reconsidering their regulation of homeopathy. We also engaged with the NIH regarding their funding of pseudoscientific research.
When it comes to influencing the FDA and CDC, I believe that Steve is probably correct, although it is important to note that SBM was alone in this effort. At various times, we did team up with the Center for Inquiry to submit comments to the relevant regulatory agencies. Even so, in particular I am proud how, as our only lawyer, routinely used to shine a light on the efforts of naturopaths and other practitioners of unscientific “medicine” to lobby state legislatures to license their pseudoscientific quackery, as well as her warning about “health freedom” bills long before they became a routine feature of the pandemic. For my part, I like to think that frequent posts by Jann and me about “right-to-try” bills (which I first likened to the movie Dallas Buyers’ Club), which emerged nearly nine years ago as a strategy to build momentum by passing “right to try” bills as state laws in order to build momentum to pass a federal law designed to weaken the FDA were prescient, even if ultimately right-to-try largely succeeded and was passed as a federal law, with the help of Sen. Ron Johnson, a name that has become very much synonymous with promoting COVID-19 misinformation and pseudoscience, as well as antivaccine legislation.
If I had to pick the single most important success of SBM it was coining and defining the term “science-based medicine” in the first place (I personally can’t take credit for that part) and introducing it into the discussion and hierarchy of medical evidence to point out how what we call the “blind spot” of EBM (i.e., prior plausibility based on basic science) and how that has been weaponized by quacks to produce not just the appearance that many unscientific, prescientific, and mystical modalities, such as acupuncture, homeopathy, reflexology, energy medicine, and the like, might actually work, but an entire pseudodiscipline of medicine, “integrative medicine.” As Mark Crislip, who this year rejoined the fold to blog for SBM after a prolonged absence, put it 11 years ago:
If you integrate fantasy with reality, you do not instantiate reality. If you mix cow pie with apple pie, it does not make the cow pie taste better; it makes the apple pie worse.
This brings us to the first lesson. The pandemic has taught us that SBM is useful for more than just approaching the claims of alternative and integrative medicine.
Ivermectin: SBM could have prevented this
There’s a line from an old song by The Clash that I now like to quote to the point of sometimes annoying readers. it comes from the 1980 song Something About England and goes:
“You really think it’s all new,””“You really think about it too,””The old man scoffed as he spoke to me,“I’ll tell you a thing or two”
The rest of the song isn’t a perfect fit to the concept that I’m trying to get across about alternative medicine and the antivaccine movement, as the old man then lectures the young man singing about his life after World War I, during World War II, and how, for all the changes that “were to come,” certain things never change at their core. However, those lines describe perfectly how there is nothing new under the sun in antivax pseudoscience or quackery. Sure, there have been superficial changes in quackery and antivax messaging since COVID-19, such as the “clot shot,” mRNA “permanently altering your DNA,” and ivermectin as a cure “they” dont’ want you to know about, but none of these are, at their core, different from old antivax messaging: Vaccine kill, vaccines somehow permanently “alter” your very essence (a message that goes back to the 1800s); and there is a safe “miracle cure” that “they” don’t want you to know about.
Interestingly, Peter Moran, one of our longtime commenters whom I used to lionize for his advocacy against cancer quackery but who over the last decade or so has become very sympathetic to the claims of unscientific medicine, said something that I agreed with:
The HCO/Ivermectin episode was a blow-by-blow replay of all the factors that have given rise to CAM. That has never been a political partisan thing.
True, Peter buried that one gem of truth in a background of denying documented connections between various right-wing astroturf think tanks and the spread of ivermectin conspiracy theories and grift. I’ll discuss that later. First, though, Peter’s comment reminded me of two that I wrote earlier this year, the first in March, when I wrote a post entitled Ivermectin: The acupuncture of COVID-19 treatments, and the second in April and entitled Science-based medicine isn’t just for CAM. The case of ivermectin shows that it never was.
I’m not going to go into a lot of detail; you can read the posts if you want the nitty gritty. The Cliffs Notes version follows, however. The reason why the comparison came to me is because the reaction of those promoting ivermectin as a highly effective treatment—miracle cure, even—for COVID-19 reminded me strongly of the approach to evidence long demonstrated by advocates of “complementary and alternative medicine” (CAM)—or, as it’s now called, “integrative medicine” or “integrative health”—and especially by acupuncture advocates. Specifically, as more and more high-quality evidence from high quality RCTs failed to find a therapeutic effect for ivermectin for COVID-19, increasingly its advocates pointed to positive studies that are less rigorous, such as observational and uncontrolled clinical studies. This is, more or less, exactly what acupuncture advocates have been doing as more and more high-quality studies with appropriate sham acupuncture placebo groups fail to find a detectable benefit for acupuncture for treating anything.
Of course, I had to concede that ivermectin was not as implausible as acupuncture from a scientific standpoint. After all, acupuncture is based on the concept of altering the flow of “qi,” a magical mystical life force “energy,” by sticking needles into “meridians,” magical mystical pathways along which qi flows that have no detectable basis in human anatomy, in order to treat diseases. Ivermectin, on the other hand, is a drug that had demonstrable antiviral activity in cell culture against SARS-CoV-2, the coronavirus that causes COVID-19. So where does the implausibility come in? After all, the drug “worked” in cell culture? Doesn’t that mean that it might actually work in animal models and then in humans?
Yes and no. Here’s the rub. As this review article documents (and I discussed in both of my posts), the the IC50 (the concentration that produces 50% of maximal inhibition of an activity) was in the 6 μM for ivermectin’s inhibition . Given that the molecular weight of ivermectin is 875 g/mol, 6 μM translates into ~5.25 mg/L or ~5.25 μg/ml, a concentration that is roughly 66-fold higher than the upper end of the range of blood levels of ivermectin safely achievable in a human’s bloodstream. Summarizing existing in vitro data, I wrote:
This is almost certainly the reason that ivermectin doesn’t work against COVID-19 in spite of its activity in vitro against SARS-CoV-2. It requires a concentration roughly 66- to 197-fold higher than is safely achievable in the blood. That’s why the review concluded that maybe an ivermectin analogue that is either more active or can achieve a higher concentration safely in the bloodstream is worth investigating. Based on the proposed mechanism in the Australian paper, ivermectin was never a good candidate as a treatment for SARS-CoV-2. As I discussed for the claim that ivermectin should be considered a promising drug for COVID-19 based on its protease inhibitor activity, the situation is just as bad. Again, based on in vitro results, ivermectin was never a promising candidate as an antiviral drug to treat COVID-19.
Or, as I put it more succinctly on Twitter:
Yup. As liked to say, it wasn't homeopathy-level implausibility, but it was highly implausible.
— David Gorski, MD, PhD (@gorskon) March 31, 2022
Also:
To accept that such a mechanism might be operative, the clinical trial results have to be highly compelling and pristine. None of the clinical trials for #ivermectin even came close to that standard.
— David Gorski, MD, PhD (@gorskon) March 31, 2022
As I elaborated, if ivermectin were to be actually useful against COVID-19, it would almost certainly have to work by a different molecular mechanism than the one described in the in vitro study cited because it’s known to be impossible to safely achieve ivermectin concentrations in human blood that are sufficient even to inhibit viral replication by 50%. While one must always concede that it’s possible that such a previously unknown mechanism is operative, albeit highly unlikely. There is a caveat, however, and Steve stated that caveat well 12 years ago, “When the basic science dictates that a proposed treatment is highly implausible, the bar for clinical evidence should be raised proportionately.” In other words, to overcome a mechanistic bar as high as that of ivermectin in cell culture, you need some really pristine, bullet-proof clinical evidence, even more so for acupuncture. As I like to put it, a highly implausible mechanism of action for a treatment coupled with equivocal clinical studies equals a treatment that doesn’t work.
I like to think that SBM (the concept, not necessarily the blog) could, if invoked properly, prevented the waste of a lot of research money on ever larger RCTs of ivermectin for COVID19 that ended up, as I predicted that they almost certainly would be, negative. Of course, SBM probably would not have helped with another aspect of this problem, conspiracy theories.
Conspiracy theories über alles
Regular readers (and just anyone paying attention) certainly realize that one of the most intractable problems for SBM (or even just EBM) during the pandemic has been conspiracy theories. Indeed, they started early. The very first time I wrote about a COVID-19 conspiracy theory was in February 2020, when James Lyons-Weiler claimed to have “broken the coronavirus code.” (Hint: He had not.) Basically, the conspiracy theory was the same one that has followed every single epidemic of a new pathogen since there have been large labs, namely that Lyons-Weiler claimed to have found evidence that SARS-CoV-2 (which hadn’t even gotten its official “final” name then) had been engineered in a lab. This was a very early version of the same conspiracy theory that persists today, the “lab leak” conspiracy theory.
I bring this up because, if there’s one thing that I think SBM needs to be better eequipped to address and counter, it’s conspiracy theories. I once coined the term “central conspiracy theory of the antivaccine movement” way back in 2014. It’s a very simple conspiracy theory that leads to ever more complex variants. The simple version is that vaccines kill or harm by various means (and don’t work), but “they” don’t want you to know about it and cover it up. This is why I wasn’t too surprised that by May 2020 conspiracy theories about COVID-19 were flourishing. (Does anyone remember Plandemic? To me it now seems like ancient history, over two and a half years ago!) Indeed, ultimately during the pandemic Steve pointed out the threat of conspiratorial thinking to SBM, and I ultimately argued that not just antivaccine beliefs and quackery were largely based in conspiracy theories, but all science denial is a form of conspiracy theory.
What the pandemic has taught us at SBM is the power of conspiracy theories when weaponized against science and medicine, which Steve recognized nearly two years ago:
This phenomenon is nothing less than an existential threat, including to science and therefore SBM. But more broadly this is a threat to any notion of evidence, logic, facts, and reality. We need to take this threat seriously, and devote time and attention to figuring it out and mitigating it.
I’ve also come to the conclusion that we will not successfully mitigate science denial of the sort that has resulted in millions of potentially avoidable deaths from COVID-19 until we are able to understand conspiracy theories and develop effective strategies for countering them because, even if you do not accept the proposition that all science denial is a form of conspiracy theory, it is without a doubt true that all science denial relies at least in part on conspiracy theories to support it. As we’ve seen in the COVID-19 pandemic, the conspiracy theory of science denial can have deadly consequences, consequences that, once the pandemic finally abates, will become apparent as the earth’s climate continues to warm due to human activity.
Ideological misinformation, disinformation, and social media
We at SBM have always noted that there were ideological entities promoting quackery and its “integration” into medicine. One just has to point out our many posts about, for example, the Bravewell Collaborative, the Samueli Institute, and any number of wealthy donors contributing to fund centers and departments of integrative medicine at academic medical centers. Compared to what SBM has to deal with now, these groups seem almost quaint.
I’m referring, of course, to groups such as the American Institute for Economic Research (AIER), which birthed the Great Barrington Declaration, which in October 2020 basically proposed a “let ‘er rip” strategy for the pandemic, with little or no mitigation to the spread of the coronavirus among the “young and healthy” and a poorly defined “focused protection” of those at high risk of severe disease and death (e.g., the elderly and those with chronic health conditions). As I said at the time, it was basically eugenicist in that it proposed sacrificing those at high risk in the name of reopening the economy and profoundly anti-public health.
Later, Jeffrey Tucker left AIER to found the Brownstone Institute, which he declared the “spiritual child of the Great Barrington Declaration.” Unfortunately, the Brownstone Institute rapidly attracted a number of ideologically similar pundits, scientists, and physicians to promote a steady stream of anti-“lockdown” rants and even outright antivaccine misinformation about COVID-19 vaccines. Basically, the Brownstone Institute is opposed to any sort of collective action in the service of public health and portrays public health efforts as, depending on their mood, Nazi or Communist totalitarianism. Increasingly, groups like this take their cue from more well-established antivax propagandists and their organizations, like Robert F. Kennedy, Jr.’s Children’s Health Defense, Joe Mercola’s longtime antivax messaging, and even at times Mike Adams.
As Steve and I (and many others) have noted, the spread of misinformation and conspiracy theories is also facilitated by the algorithms that drive social media. This is not a new phenomenon. For example, various SBM bloggers were discussing research about how antivaxxers were gaming Facebook against vaccine advocates years before the pandemic and how inadequate the efforts of social media companies were to the task, but these days it’s arguably worse than ever.
Worse, unlike past the misinformation and disinformation about vaccines and pandemics (e.g., the H1N1 pandemic 13 years ago), it’s not just ideological entities promoting antivax and COVID-19 misinformation, but what would be considered “mainstream” outlets, such as Fox News, particularly Tucker Carlson, who routinely parrots common antivax lies about vaccines while lamenting the apparent decline of men.
This brings us to an elephant in the room that the role of SBM in promoting the best medical evidence has traditionally made us reluctant to emphasize too much, namely the partisan imbalance among those promoting COVID-19 misinformation.
Political vs. partisan
We at SBM have never claimed to be apolitical. How could we credibly make such a claim when we regularly take what are inherently political positions with respect to, for instance, opposing right-to-try, licensing naturopaths and other quacks, vaccine mandates (prepandemic and now), and how the FDA licenses new medications? Here’s how I like to characterize us. SBM is not apolitical, but we do strive to remain nonpartisan and to promote political policies and regulations that are based in and promote a high standard of scientific evidence for medicine. Indeed, just search for Senator Tom Harkin’s name, given that he was the driving force behind the creation of the National Center for Complementary and Alternative Medicine (NCCAM), which is now known as the National Center for Complementary and Integrative Health (NCCIH) and a Democrat who once famously expressed his disappointment that NCCAM hadn’t “validated” more CAM. Back then, the stereotype was that the antivaccine movement and believers in alternative medicine consisted mainly of crunchy hippie-dippy lefties.
Years ago, I routinely used to point out that survey evidence indicated that support for vaccination and school vaccine mandates was pretty similar on the right and the left. (At least, it was then.) I also used to suggest that, back in Jenny McCarthy‘s heyday as the celebrity face of the antivaccine movement 15 years ago, it was the heavy representation of celebrities among famous antivaxxers that contributed to the public perception that the antivaccine movement was predominantly left wing, Hollywood celebrities like Rob Schneider (admittedly, with him I’m probably being generous in my definition of “celebrity”), Mayim Bialik, Alec Baldwin, and Robert De Niro, among others. It’s also true that areas with a lot of affluent people on the coasts, whose politics tend to lean heavily liberal, used to routinely suffer outbreaks of vaccine-preventable illnesses in recent years leading up the pandemic due to low vaccine uptake. Then, of course, there was (and, alas, still is) Robert F. Kennedy, Jr.
To be sure, there was always such a contingent of a “back to nature” crowd, but in reality that stereotype was never true. For example, there has always been a libertarian right wing component to the antivaccine movement, for example, General Bert Stubblebine III’s Natural Solutions Foundation, far right libertarians, and others with extreme distrust of the government, including government-recommended vaccine schedules. Similarly, the “health freedom” movement has tended to be right wing and also very antivaccine.
The elephant in the room, therefore, is just how much the antivaccine movement has found a home on the far right and how much COVID-19 disinformation comes from the political right as compared to the left. This is not a new phenomenon, but rather a culmination of a drift that began over a decade ago (at least). For instance, as early as 2011, Fox News was airing sympathetic segments on Andrew Wakefield, interviews with Dr. Bob Sears, SafeMinds’ anti-vaccine PSA campaign, and Louise Kuo Habakus (who was virulently anti-vaccine herself and politically active in New Jersey advocating for more easily obtained “philosophical exemption” laws). Let’s also not forget about Rep. Dan Burton (R-IN), who routinely used to hold “hearings” before his committee about the vaccine-autism link back in the 1990s and early 2000s.
Then there was The Canary Party, founded in 2011, which soon started working with Tea Party-affiliated groups in California. Not long after, the Canary Party became known for sucking up to Rep. Darrell Issa (R-CA), with one of its major financial backers Jennifer Larson contributing a lot of money to Issa’s campaign (indirectly, of course) in order to buy influence and win a hearing by his committee examining autism and focused on vaccines as one potential cause. Around the same time, at the right-wing Libertarian FreedomFest in 2012 I was privileged to watch a debate between Julian Whitaker and our very own Steve Novella about vaccines. At the debate, vaccine pseudoscience flowed freely from Whitaker in a most embarrassing fashion, and I couldn’t help but note that FreedomFest that year featured two screenings of Leslie Manookian’s antivaccine propaganda piece, The Greater Good and had featured antivaccine talks in previous years. I was there, too, and amazed at the merchandise and conspiracy theories being touted, although in retrospect, in the era years before the rise of QAnon, conspiracy theories about the gold standard and New World Order now seem almost quaint.
This leads me to something else that Peter Moran said:
Antivax sentiments and mandate resistance have never required political underpinnings.
And:
Perhaps familiarity — sheer repetition in a nurturing tribal environment — alongside a little political bias can give such nonsense a veneer of truth.
The first part is sort of true but not really. What do I mean? Unless you consider politics separate from ideology, it’s hard to claim that antivax sentiments are independent of political underpinnings, because antivax sentiments are very much tied to ideology. The second part is also likely true to some extent, but not in the way Peter intended. He was mocking the idea (“such nonsense”) that a network of right wing groups are supporting the spread of COVID-19 and antivax disinformation, when there is actually good evidence to support this contention. In fact, it’s definitely likely that “sheer repetition in a nurturing tribal environment” can give misinformation, be it antivaccine misinformation or misinformation about COVID-19, or just general quackery, can do just that. We were writing about how this could happen long before the pandemic, such as the story of Jim Laidler as well as the appeal of being a medical “apostate.”
Moreover no one here at SBM is claiming that the left is not capable of promoting medical misinformation. Certainly it is. The problem is, as I like to say, that right here, right now, at the end of 2022, it is inarguable that the primary forces promoting antivaccine, anti-public health, and COVID-19 disinformation are currently right wing, often far right wing, to the point where public health responses to COVID-19, particularly vaccines, have become very partisan, a process that began long before the pandemic, when antivax ideology reared its ugly head during the 2016 Republican primaries and in 2018 when it showed up in the form of a “roundtable” held by the local Republican candidates in my district, and now has Sen. Ron Johnson holding antivaccine “round table” discussions featuring a bunch of quacks and grifters and the governor of a large state not only using his state public health apparatus to promote studies questioning vaccines but openly calling for a mini-“Nuremberg 2.0” (although in fairness he didn’t call it that).
Basically, what this politicization means is that, no matter how much we try to assiduously remain nonpartisan, just by advocating for interventions and policies rooted in science-based medicine, we are no longer going to be able to avoid charges of being partisan, “woke,” or whatever. This is a problem that we struggle to deal with, particularly given that other areas of SBM are similarly becoming partisan in a way that we haven’t had to deal with before, areas such as gender-affirming care for transgender adolescents. The problem, then becomes that our message will be rejected out of hand based solely on the perception that we are somehow far left wing and highly “woke” (whatever that word even means anymore).
I would be lying if I didn’t say that I’m struggling how to deal with this development, mainly because the only way to be seen as non partisan would appear to be to soft peddle our advocacy for SBM-based policies in certain areas, which we will not do. Yet, at the same time we need to reach everyone who is reachable, and the fusion of COVID-19 quacks and antivaxxers with right wing political movements makes it a huge challenge, one that I’ve experienced in my personal life trying to convince conservative acquaintances who, as I like to say, came to COVID contrarianism for the freedom but stayed for the conspiracy theories and pseudoscience. In addition, there is no way of knowing if someday the partisan landscape might change someday (although how it would change any time soon I struggle to envision), which means that we have to be nimble and consistent.
The best solution that I can think of for now is to promote the best science that we can and to pull no punches when criticizing those promoting pseudoscience, quackery, and conspiracy theories, regardless of their ideology or partisan politics. At the same time, we have to try not to alienate those who might be predisposed to embrace the misinformation that is associated mainly with one partisan side of the political spectrum, regardless of the politics of me or any SBM blogger. It’s a balancing act that is at best difficult and that might well be impossible. How do we avoid doing nothing more than “preaching to the choir”? I argue that “preaching to the choir is not in and of itself a bad thing given that it provides science advocates tools to communicate science and counter medical misinformation, but it clearly isn’t enough by itself.
SBM and the future since the pandemic
The quackery and medical misinformation landscape now, compared to when SBM debuted 15 years ago, is very different and yet the same. As I wrote at the dawn of our now fading year, everything old is new again, with the sarcastically added question, “Who could possibly have predicted this?” It’s the same in that there is nothing new under the sun. Misinformation about medicine, quackery, and vaccines follow an old script that dates back decades, if not centuries: “Health freedom,” appeals to the “natural,” the idea that one can will oneself to health, and distrust of medicine and political authorities. I refer to my post a year ago for the details, for those who are interested.
The first step in combatting this threat remains the inculcation of critical thinking skills into the next generation. Obviouslym this is easier said than done, and we need to explore strategies for doing this not just in medical school but much earlier, as it’s not just physicians who need it. I would also argue that SBM as currently constituted needs help. Our hardy band of bloggers have done the best we can and have tried to recruit guest bloggers to fill in the gaps, but gaps remain. Besides COVID-19, we lack high level expertise in women’s health, psychiatry, epidemiology, and a number of medical subspecialties. Guest posts are always welcome in these areas, but we could really use regular contributors to join us.
We also need to emphasize science communication more, in particular “prebunking.” Increasingly, the combination of social and old media result in “firehosing” of misinformation, in which falsehoods and misinformation fly so fast and furious that it’s impossible to counter them all. We need to be more proactive about educating about the forms that misinformation takes, such as the conspiracy theories at the root of nearly all of it, and why it is being promoted, in order—if you will—to inoculate people to be able to recognize new forms of misinformation or claims that are questionable based on the forms that they take, even if the person doesn’t necessarily have the detailed knowledge to know specifically what is not accurate in the claims.
Finally, I’m not getting any younger. As hard as it is for me to believe, I just turned 60 recently. If SBM still exists and I’m still with it 15 years from now, I’ll be 75 when our 30th anniversary rolls around. As much as I’d like to pretend otherwise, I can’t do this forever, and neither can Steve, who is younger than I am but not that much younger. We need to nurture the next generation, so that someday we can hand over the reins to others to continue the mission. The problems of quackery, medical misinformation, and conspiracy theories have long been with us and are not going away any time soon. The mission of SBM will need to continue after Steve and can no longer do this. “You really think it’s all new,” indeed.
In the meantime, sometimes we do feel as though we are trying to rouse the troops, like Aragorn at the Black Gate of Mordor, to fight what seems like a hopeless battle.

2023: The year that the evidence-based medicine (EBM) paradigm was weaponized against vaccines and public health
Evidence-based medicine (EBM) has been a very useful paradigm for assessing evidence in medicine. However, like any other framework, it can be misused, particularly when fundamentalist EBM methodolatry leads to its inappropriate application to questions for which it is ill-suited, a misuse that has been weaponized against public health during the pandemic.
David Gorski on January 1, 2024
Tweet

Since the COVID-19 pandemic arrived in 2020, my posts on or about New Years Day each year started to consist of my ruminations about various trends and tactics in science and medicine denial that I had noticed during the prior year. For instance, as the first year of the pandemic was shambling to a close, I noted how 2020 had been the year of so many physicians behaving badly in terms of promoting COVID-19 misinformation and outright quackery. Then, as the second year of the pandemic lumbered into the third, I pointed out how 2021 had become the year when the weaponization of the VAERS database to portray COVID-19 vaccines as deadly, something that antivaxxers had been doing for a long time prior to the pandemic to falsely portray childhood vaccines as a cause of autism and a number of other health complications they do not cause, had become mainstream, thanks to the pandemic. As 2022 faded into 2023, I then pointed out how in 2022 antivaxxers and other anti-public health activists had decided to claim that the public was being “gaslighted” about COVID-19. Of course, all of these trends were nothing new, because, as I repeat—at times ad nauseam—when it comes to quackery, science denial, and conspiracy theories, there is nothing new under the sun and everything old is new again. The pandemic just supercharged tactics that cranks, quacks, and antivaxxers had been using for years, if not decades, and, as newbies discovered them and started using them to build influencer audiences and/or promote a certain political orientation, migrated alarmingly to the mainstream.
As difficult as it is to believe now, March will usher in the fourth anniversary of the World Health Organization’s (WHO) declaration that the outbreak due to the novel coronavirus that had been given the name COVID-19 was officially a pandemic, meaning that we will soon be entering the pandemic’s fifth year. So what trend did I notice in 2023, the fourth year of the pandemic? The title of this post, of course, gives away the answer. Increasingly, a simplistic use of the evidence-based medicine (EBM) paradigm has been weaponized against not just pandemic-era public health nonpharmaceutical interventions (NPIs) like masks and social distancing to give a false impression that there is no evidence that any of these things work to slow the spread of COVID-19 or even just against COVID-19 vaccines. No, increasingly, misuse of the EBM paradigm to spread fear, uncertainty, and doubt has spread from the antivax crankosphere to seemingly “respectable” EBM mavens. Some of them (for instance—cough, cough—Vinay Prasad) have even started parroting a narrative “pioneered” by Aaron Siri, the lawyer for one of the granddaddies of the 21st century antivax movement, Robert F. Kennedy, Jr. (RFK Jr.), in particular his claim that existing childhood vaccines have not been properly tested in randomized double-blind saline placebo-controlled clinical trials. Worse, supposedly “respectable” physicians parroting this sort of message have been influencing medical students, one of whom seems to think that there is an elite RCT strike force out there able to do randomized clinical trials to answer every question about a fast-moving pandemic.
Of course, the reason presented for appealing to the EBM paradigm and RCTs (and, of course, meta-analyses of RCTs) as the highest form of clinical evidence in the EBM paradigm is presented as a value that everyone in science and medicine claims to prize: Raising the standard of evidence. After all, who could possibly argue with wanting more and higher quality evidence to support what we do in public health and medicine? Isn’t this entire blog dedicated to the very concept that the bar needs to be raised for the standard of evidence, that more science is needed? Absolutely! There’s just one problem (well, more than one actually). What I’ve increasingly been calling EBM fundamentalists argue is that raising the bar of evidence necessarily requires RCTs for practically everything, because the EBM paradigm tells them that RCTs are the highest, most rigorous form of clinical evidence. There is good reason to question that assumption at the heart of physicians casting doubt on public health interventions because there haven’t been RCTs that they view as adequately powered and rigorous testing the efficacy of these interventions and even better reason to question the —or even sometimes outright argument—that no intervention should be used until it has been validated in large RCTs, almost regardless of the practicality or ethics of doing such trials.
Long ago, as another pandemic (H1N1) bore down upon us, I learned a great term: Methodolatry. The term describes a phenomenon when the method matters more than the outcome and was sarcastically defined by a very senior epidemiologist and blogger who wrote under the pseudonym “revere” as “the profane worship of the randomized clinical trial as the only valid method of investigation.” For my part, I sometimes define EBM methodolatry by referencing an old Mike Myers sketch on Saturday Night Live about a Scotsman who frequently said, “If it’s not Scottish, it’s crap!” (There was always a huge emphasis on the word “crap.”) In doing impressions of these EBM methodolatrists, I would just substitute “an RCT” for “Scottish,” because it gives the general idea behind the misuse of the EBM paradigm, which promotes the idea that if it’s not an RCT it’s not evidence—or at least it’s not evidence not sufficiently rigorous to be even worthy of consideration in coming to a decision about how to deal with a medical or public health issue. Going back over revere’s old post, I noticed this comment about the article by Cochrane Collaboration scientist and physician Tom Jefferson—who is these days using similar tactics to cast doubt on masks and COVID-19 vaccines—casting doubt on the efficacy of the flu vaccine that had provoked revere’s ire:
My specific point is that we have here a concrete example of the misuse of EBM that is having real world effects, perhaps to the point that one or more people will die from novel H1N1 because this article convinced them the benefit of getting inoculated is so low it doesn’t outweigh the perceived risks.
Sound familiar? Let me just mention that the post that I am citing was published over 14 years ago. Indeed, you could take a number of articles citing the lack of RCTs about this NPI or for COVID-19 boosters or for a number of issues that have come up during the pandemic and change the words “novel H1N1” to “novel coronavirus” or “COVID-19,” and the point is just as valid today as it was in 2009. The only difference today is that far more EBM gurus seem to be promoting the same simplistic and misleading message that Tom Jefferson was back then; worse, the same idea is metastasizing from COVID-19 to the childhood vaccination schedule. Moreover, I would argue that this common pandemic-era misuse of the EBM paradigm is very much related to the “blind spot” for alternative medicine quackery that we’ve discussed here going back to the very beginning of this blog that allows quackery like homeopathy to be “integrated” into medicine in what used to be called complementary and alternative medicine (CAM) but is now more commonly called “integrative medicine” or “integrative health” and, more recently, the widespread promotion of ineffective drugs for COVID-19 like ivermectin and hydroxychloroquine that led me to liken ivermectin to acupuncture.
So let’s dive in deeper. First, of course, rather than just linking to prior posts or other sources, I will briefly recount what the EBM paradigm is, because you need to understand that before I can explain how some have twisted the EBM paradigm into methodolatry in order to spread the message that there is “no evidence”—yes, some of them use the term—that, for example, masks work.
The EBM paradigm
The EBM paradigm is often described in terms of the “evidence pyramid,” in which types of scientific evidence applied to medicine are ranked thusly:
EBM pyramid of evidence. Note that anything that is not an RCT is automatically considered “lower quality” evidence and that the highest form of evidence of all is meta-analyses and systematic reviews of RCTs.
The “classical” way of looking at levels of evidence from clinical studies not involving the synthesis of comes in the form of levels of evidence, as originally proposed by D. L. Sackett:
Level	Type of evidence
I	Large RCTs with clear cut results
II	Small RCTs with unclear results
III	Cohort and case-control studies
IV	Historical cohort or case-control studies
V	Case series, studies with no controls
(Adapted from Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest 1989;95:2S–4S.)
Randomized controlled clinical trials are just that: clinical trials in which subjects are randomized to receive either the intervention or a control (such as a saline placebo) and then followed to determine which group has better outcomes, which are prespecified in the clinical trial protocol. The reason for randomization is to ensure that the two groups being compared resemble each other as closely as possible in characteristics relevant to the outcomes being tested. For example, if you’re testing a drug to treat hypertension, you would want the groups to be matched as closely as possible for, among other characteristics, age, race, sex, severity of hypertension, and relevant risk factors for poor outcomes. Ideally, these RCTs are then double-blinded, so that neither the subjects nor the doctors or medical personnel administering the drugs and assessing outcomes know which group any given subject is in. Double blinding is especially important in clinical trials with more subjective outcomes such as pain, for which placebo effects can be strong, but it’s also important even in trials with “hard” outcomes like tumor progression because it could affect how clinicians interpret tests and radiology studies if they know which group a given patient is in. Moreover, such clinical trials have strict inclusion and exclusion criteria, which ensure that those being treated actually have the disease, do not belong to a group that might be harmed by the drug, and are subjects who are likely to benefit if the drug does have efficacy; i.e., does work.
As you can see in the image above, in the EBM paradigm RCTs are ranked near the top of the pyramid for strength of evidence. Actually, as far as original clinical go, they are at the top. It is true that systematic reviews and meta-analyses of RCTs are ranked higher (i.e., as more rigorous), but remember that these are syntheses of existing RCTs, not original experimental clinical studies in which one intervention is tested against placebo or another intervention. Moreover, it is also true that, for certain questions, RCTs truly are the gold standard, as an RCT is the design that best minimizes bias, thanks to randomization and careful matching of the experimental versus the control group, plus blinding, which when done properly minimizes bias due to observation. None of us at SBM has ever denied that RCTs are indeed usually the most rigorous method for determining if a treatment works, and far be it from us to advocate for a lower standard of evidence. However, we have pointed out problems with the EBM paradigm.
Let me discuss one specific example. One problem with the EBM paradigm is that, because it values RCTs above all else, including huge quantities of epidemiological evidence, any evidence not from RCTs will automatically be relegated to a lower rung of certainty, no matter how copious the evidence is, how plausible from a basic science standpoint, or from how many disparate sources the evidence comes, all coming to the same conclusion. For example, let’s take a look at what the gurus of EBM at Cochrane have to say about the question of whether the MMR or MMRV vaccine causes autism. (Hint: Neither of them does.) Its review is Does the measles, mumps, rubella and varicella (MMRV) vaccine protect children, and does it cause harmful effects? Let’s see what it says.
While the review does conclude that there is “no evidence” that the MMR or MMRV combination vaccine causes encephalitis or autism, it does say:
Our certainty (confidence) in the evidence is slightly limited by the design of most of the studies. Nonetheless, we judged the certainty of the evidence for the effectiveness of the MMR vaccine to be moderate, and that for the varicella vaccine to be high. Our certainty in the evidence for autism and febrile seizures was also moderate.
Regular readers of SBM likely already know that in reality the evidence base of studies examining the question of whether vaccines cause autism is very large and has failed to find even hint of a whiff of an association between MMR (or MMRV) and autism. Yet, because the bulk of these studies consists of epidemiology, including one that encompassed 537,303 children representing 2,129,864 person-years of study, the certainty of the evidence behind Cochrane’s conclusion that MMR does not cause autism can never be more than “moderate” because by the very definition of the EBM paradigm and its ranking of sources of evidence, epidemiological studies always rank below RCTs. Note that it will never be possible to do an RCT to answer the question of whether vaccines like MMR increase the risk of autism because such a trial would be inherently unethical, as it would require randomizing a large number of children to the placebo control group, leaving them unprotected against a disease, a vaccine against which is considered standard-of-care for prevention of this disease; in other words, randomizing them to a group where we know that they would suffer harm. Moreover, such an RCT would require inordinately large numbers of subjects followed for such a long time as to be utterly impractical, as defenders of science-based vaccine requirements were explaining 15 years ago.
Another way to look at this is to examine the more recent version of the levels of evidence published by the Centre for Evidence-Based Medicine:
Levels of Evidence for Therapeutic Studies*
Level	Type of evidence
1A	Systematic review (with homogeneity) of RCTs
1B	Individual RCT (with narrow confidence intervals)
1C	All or none study
2A	Systematic review (with homogeneity) of cohort studies
2B	Individual Cohort study (including low quality RCT, e.g. <80% follow-up)
2C	“Outcomes” research; Ecological studies
3A	Systematic review (with homogeneity) of case-control studies
3B	Individual Case-control study
4	Case series (and poor quality cohort and case-control study
5	Expert opinion without explicit critical appraisal or based on physiology bench research or “first principles”
*From the Centre for Evidence-Based Medicine, http://www.cebm.net.
Under this paradigm, even systematic reviews of epidemiological studies are automatically at best level 2 evidence, at least when it comes to therapeutic studies. I do realize that there are other evidence rankings that put epidemiological evidence higher when it comes to questions of etiology or association. For example, it is noted here:
Since the introduction of levels of evidence, several other organizations and journals have adopted variation of the classification system. Diverse specialties are often asking different questions and it was recognized that the type and level of evidence needed to be modified accordingly. Research questions are divided into the categories: treatment, prognosis, diagnosis, and economic/decision analysis
Also:
In the absence of a current, well designed systematic review is not available, practitioners turn to the primary studies to answer their questions. The best research design depends on the question type. The table below lists optimal study methodologies for common types of clinical questions.
With a summary chart:
Question Type	Optimal Study Methodology
Treatment (Therapy) & Prevention	Randomized controlled trial (RCT)
Diagnosis	Randomized controlled trial (RCT)
Crossover study: Subjects receive both the intervention (diagnostic test) under investigation and the reference standard or “gold standard” diagnostic test
Prognosis(Natural History)	Cohort study
Etiology or Harm(Causation)	Randomized controlled trial (RCT)
If RCTs are impractical or unethical, cohort study or case control study
I note that EBM fundamentalists frequently ignore the precept that “if RCTs are impractical or unethical,” then cohort and case control studies must do. However, notice how, even the assessment of what is the optimal study design for each type of question only accepts non-RCTs for questions of etiology or harm (causation). In reality, questions of public health during the pandemic involve interventions, treatments, and/or preventatives, such as masking, social distancing, and vaccines.
The bottom line is that the EBM paradigm being applied to COVID-19 interventions was designed for a fairly narrow purpose: To rank the evidence primarily dealing with pharmaceutical drugs used for specific indications, in particular fairly simple, straightforward indications. Where the paradigm struggles is when it is applied to more complex, multifactorial conditions and questions. In particular, it struggles when applied to public health. Worse, it can be misused, and, unfortunately, a lot of EBM “gurus” seem predisposed to fall into the trap of doing so.
EBM misused a tool to attack public health interventions
Although I had been mulling the idea of how EBM had been weaponized against public health interventions and had started to write this article, I knew I had to complete it when I saw that Dr. Vinay Prasad had decided to finish 2023 with an article entitled Hospitals that reinstate universal masking with flimsy surgical masks are anti-science. And why is that? Because not “even one hospital is running a cluster randomized trial.” It’s a rather short rant (for Dr. Prasad) that concludes with:
Masking has made a fool of most scientists. They love bioplausible stories, but have failed to conduct even the most basic efficacy trials. Masking in hospitals is unlikely to slow the juggernaut of respiratory viruses— both iatrogenic and the more relevant q of across the community. Implanting the policy year over year— and running 0 cluster randomized trials— is embarrassing for medicine. We might as well open up a Vitamin C infusion center and pray to the sun god.
Actually, as I’ve argued before, insisting on RCTs for everything, regardless of whether RCTs are the most appropriate method to investigate a question or of how impractical or, in some cases unethical, an RCT might be has made a fool of fundamentalist EBM methodolatrists like Dr. Prasad and Dr. Tom Jefferson of Cochrane. While it is true that cloth masks are not great for this purpose, we do know from various reviews (and even reviews of systematic reviews) of what Dr. Prasad would consider “inferior” studies that N95 masks are effective in reducing transmission among healthcare workers and that masks in general can decrease transmission. It is not “antiscience” to say that, nor is it “antiscience” to question whether doing RCTs on masks of the type that Dr. Prasad is even ethical anymore given the current body of “lesser” evidence.
I will admit that Dr. Prasad’s little sarcastic dig at vitamin C infusions and praying to the sun god did lead me to chuckle, but not for the reason that Dr. Prasad no doubt intended. Remember before the pandemic, when Dr. Prasad was making fun of those of us who had spent so much time deconstructing quackery like homeopathy as having wasted our time and intellects fighting misinformation that, to him, was so obviously ridiculous that it was a waste of time compared to doing what he did? He even characterized it as “dunking on a 7′ hoop.” Yet, he clearly doesn’t understand that refuting the misinformation about, for instance, vitamin C infusions as a treatment for cancer is not at all straightforward.
If you want to get an idea of just how much EBM fundamentalism and methodology have polluted Dr. Prasad’s mind, remember that he was partially down with a favorite antivax trope that has been promoted by Robert F. Kennedy, Jr. and the lawyer for his antivaccine organization Children’s Health Defense, Aaron Sir, namely the claim that most of the vaccines in the CDC’s recommended childhood vaccine schedule had never been tested in an RCT versus saline placebo, the implication being that their safety and efficacy were in doubt. Indeed, let’s take a look at him falling for the “no saline placebo” RCT gambit:
Yet on hepatitis B vaccination, RFK Jr makes a fair point that we could target higher risk populations. At least that could be the central question of a randomized control trial. He’s also fair to ask if the particular timing of doses is optimized, or if it could be given it a later date. That could also be a randomized trial. I think this century will see the pediatric childhood immunization schedule the subject to a multi-arm, factorial RCT. It is better to launch that effort now before public distrust gets too out of control.
As I pointed out, this is a longstanding antivax trope to call for RCTs of the childhood vaccination schedule, despite the expense and the impracticality of such an effort. Dr. Prasad’s suggestion was nothing new, other than making the call into a “multi-arm, factorial RCT.” Indeed, at the time I challenged Dr. Prasad to design such a “multi-arm factorial RCT” of the childhood vaccination schedule that would be (1) feasible; (2) not require hundreds of thousands or millions of participants; and (3) ethical. Remember, any placebo-controlled RCT would require leaving one group unprotected against potentially deadly infectious diseases. I also pointed out how the antivax hatred of the birth dose of the hepatitis B vaccine is based more on moral outrage that leads them to rage that, because my baby or child isn’t at risk because my child doesn’t have premarital sex, share needles, or engage in what I consider morally dubious high risk behavior, my child shouldn’t need the birth dose of hepatitis B vaccines (or the vaccine at all). The subtext, of course, behind saying that the “high risk” groups should be targeted is that those “dirty vaccines” should be reserved for people who need them because they are dirty too, just as the call for only “high risk” people to be vaccinated against COVID-19 was based on the false believe that those who “lived right” and are “healthy” don’t need protection against the disease.
After having played footsie with RFK Jr.’s bonkers antivax conspiracy theories about the childhood vaccination schedule, Dr. Prasad didn’t take long to double down by writing The only real solution to vaccine skepticism: We need more randomized trials and a new phase IV safety system. Its blurb even led me to facepalm: “The way to address concerns about vaccines and other medical products is to hear what people are saying.” I discussed in detail how, in the name of winning back “trust,” Dr. Prasad doubled down on EBM fundamentalism and demanded RCTs of practically everything vaccine-related and how he was parroting old antivax tropes, but let’s just look at this one example here:
Trials of vaccine should contain at least 2 control arms. One a placebo arm of salt water, and another placebo arm perhaps containing adjuvant/ preservative/ or a different vaccine. Each control arm has different strengths and weaknesses. One allows accurate assessment of safety; the other preserves blinding, and or downstream behavioral change (think about it for a while).
Here we go again. Dr. Prasad is echoing yet another antivaccine talking point, namely the “no true saline placebo control RCTs” trope about vaccines. First of all, ethics, how does that work? As I discussed the other day, while it is appropriate to insist on a placebo-controlled RCT for a new vaccine against a disease for which there is currently no safe and effective vaccine licensed and available, it is completely unethical to do such a trial for a new vaccine against a disease for which there already exist safe and effective vaccines that are the standard of care. You cannot ethically randomize subjects to an arm where they receive less than standard, accepted medical care, such as randomizing a child to an arm where, for example, he doesn’t receive MMR because you want to test a new combination measles vaccine. The ethical thing to do is to test a new next generation vaccine against the existing standard-of-care previous generation vaccine for the same disease. This is methodolatry on top of methodolatry in that, not only is the RCT the only acceptable method of clinical investigation to Dr. Prasad, but it has to be a specific kind of RCT with a specific kind of placebo, clinical trial ethics and Helsinki declaration be damned!
But Dr. Prasad should know this. He’s an oncologist, and oncology trials were the type of RCTs that I used as an example to illustrate why placebo controls are often not ethical. Ethical RCT design is why most new chemotherapy agents used in oncology are either tested in an add-on design (standard of care + new drug versus standard-of-care + placebo) or a design that uses an active comparator (new drug vs. standard-of-care), so as not to require the randomization of subjects to an arm in which they receive no treatment for their cancer. Again, I’ve discussed this a number of times before. Again, saline placebo control is not the only appropriate placebo comparator. Adding it as a third arm in an RCT using a more appropriate comparator when that comparator has a long, well-established history of safety, with well-known and well-characterized potential adverse events, would do nothing but add expense and complexity to the clinical trial, as well as necessitate larger numbers of subjects in the phase 3 trials, all for no real benefit. Does Dr. Prasad honestly think that retesting existing vaccines in new RCTs in which there is a saline placebo control arm would reestablish “trust.” If he does, he doesn’t know the antivaccine movement, which would simply shift to another of its many talking points and/or find reasons to discount the new trials.
I could pick on Dr. Prasad for much longer, but unfortunately he’s not the only EBM fundamentalist up to such shenanigans. For example, Peter Götzsche has been abusing the interpretation of RCTs to portray COVID-19 vaccines as harmful, adding to his history of having invoked the “no saline placebo control” gambit before the pandemic to try to do the same about HPV vaccines. (Indeed, even before the pandemic Gøtzsche played footsie with antivaxxers so much that he only backed out of a speaking engagement for an antivax physicians’ group at the last minute because of embarrassment.) Indeed, such methodolatry seems to be a characteristic of EBM fundamentalists. Another example includes Tom Jefferson, who invoked similar gambits to claim that the entire childhood vaccination schedule is unscientific and unproven.
Worse, as Dr. Jonathan Howard has pointed out, there is now a whole cadre of physicians demanding RCTs for every COVID-19 intervention, regardless of whether such RCTs are practical or even ethical to carry out. Not coincidentally, he also noticed that the same physicians were often content to cite non-RCT evidence for purported harms of vaccines and COVID-19 interventions, a clear double standard. For example, Dr. Prasad might be well-known for demanding ever more rigorous data for various medical interventions, such as masking to slow the spread of COVID-19, but on the topic of myocarditis and COVID-19 vaccines, he’s long seemed quite happy with a low-quality study that misused the VAERS database. He also seemed quite happy doing a commentary about “obsessive criticism” of him and his buddy Prof. Ioannidis social media based on even lower quality analyses. Worse, this misuse of EBM seems to be influencing trainees, such as medical students.
Indeed, calling for a bunch of RCTs that can’t be done is the point, as the goal is not, as claimed, to advocate for a higher standard of evidence but rather to cite the existence of impossible studies not meant to be taken seriously as a tactic to cast doubt on the evidence base for interventions against COVID-19 that they don’t like, such as masks and vaccine mandates. If you doubt me, just consider how Dr. Prasad has embraced the antivax message of “do not comply” to any intervention (especially vaccines) that he considers to have insufficient evidence, meaning (by EBM definition) insufficient or no RCT evidence. I once asked whether EBM fundamentalism contributed to COVID-19 contrarianism. I’m forced to conclude reluctantly that the answer is yes.
What do I mean by fundamentalism? This concluding paragraph of a rant by Dr. Prasad about how nobody seems to follow EBM anymore sounds almost religious in its fervor:
Evidence based medicine is delicate, vulnerable, but also profoundly beautiful and rational. It alone is the logical way to implement and pursue costly, invasive, intrusive, and disputed medical and public health interventions. As such, it is a forced move in human development. Even if it withers and dies in the next few years— and it is well on its way to demise— it will be reborn and rekindled in a future moment. It the telos of progress in science. If only the stars could see that.
I mean, seriously? EBM dying and rising again? Get a grip on yourself. EBM is just a tool, a framework to evaluate evidence, and like any other tool or framework it has strengths and weaknesses. In its current form, EBM’s key strength, where it performs the best, is in the evaluation of pharmaceutical drug treatments for well-characterized medical conditions and diseases, in particular as a tool to determine if a new drug is sufficiently effective and safe to be licensed by regulatory agencies for sale. No one, least of all I, would propose replacing EBM with something else for this purpose. However, when it comes to whole swaths of medicine, particularly the “integration” of alternative medicine into medicine as CAM or “integrative medicine,” it has serious deficiencies. That was the whole reason that Steve Novella and colleagues proposed SBM instead and started this blog 16 years ago. It is the reason why I pointed out how SBM could have prevented one waste of resources with respect to ivermectin as a treatment for COVID-19 when EBM demanded more and more RCTs before it would conclude that ivermectin doesn’t work. Now it’s being used to undermine confidence in what has long been considered as close to settled science as there can be, childhood vaccines.
The problem is that, like all fundamentalism, EBM fundamentalism refuses to acknowledge the serious shortcomings of its god that need to be addressed and fixed. Sure, methodolatrists like Dr. Prasad will concede some relatively minor shortcomings, but for him these shortcomings can only be fixed by the ever more zealous application of EBM principles—but only when it comes to addressing questions whose answers thus far he doesn’t like. How is that any different from the ideology that EBM fundamentalists accuse critics of? Remember, an accusation like that is very often a confession. Again, EBM is a tool, and it is not “antiscience” to point out where it falls short or suggest ways in which it might be improved.

SBM in 2023
Some thoughts as we complete our 15th year of science-based medicine.
Steven Novella on December 20, 2023
Link

This will be my last post for 2023, and as always I like to look back at SBM over the past year, and look forward to what is to come.
2023 was a sad year for SBM because we unexpectedly lost one of our founding editors, Harriet Hall. She was a tireless defender of science and reason in medicine, and her contributions have been sorely missed. Over the years she contributed hundreds of posts to SBM and was a welcome addition to every conference we attended.
SBM forges ahead into our 16th year. Mark Crislip rejoined the fold in 2022 after taking a hiatus. We missed his particular brand of science-based sarcasm and are glad to have him back. We continue to have a strong backbone of regular contributors, including Clay Jones, Jonathan Howard, and Scott Gavura, in additional to many contributing authors (too many to name, but thank all of you). We also have some regular contributors with a formal or informal series of posts, such as Frank Han and AJ Eckert. And of course our managing editor, David Gorski, in additional to anchoring our lineup every week, does an excellent job of pulling it all together. Behind the scenes, Ian Callanan (working with me and Jay Novella through the New England Skeptical Society) manages the back end and our hosting.
While this is a strong lineup, we are always looking for new either one time, topic-specialist, or regular contributors. The more voices we have the better. We welcome submissions from everyone – just see our submissions guidelines above – and have been happy to have many high quality submissions. Being a regular contributor is a serious commitment, which is why I am so grateful to everyone who has given so much of their professional time to this project.
Working with SBM is also a great way to work with experienced editors in an area that requires specific expertise – in science communication, critical thinking, scientific literacy, media savvy, and medical pseudoscience. We desperately need to cultivate these skills within all professionals, and arguably in medicine the need is most acute. This is a never-ending project, and we are especially keen to develop the next generation of SBM advocates and communicators.
The topics we have most had to deal with over the past year have been fairly typical. The same pseudoscience “headliners” remain our prime targets. Acupuncture continues to gain ground within mainstream medicine, despite being devoid of any coherent mechanism or theory, any internal validity, or the slightest compelling evidence of efficacy. It is the best example of the emperor having no clothes in medicine that I can think of.
Except, perhaps, for homeopathy – the “air guitar” of medicine. Homeopathy, despite being just magic water, continues to be a multi-billion dollar industry that has managed to get a free pass from most regulators. It perhaps benefits the most from the “knowledge deficit problem” – most people don’t know what homeopathy actually is. This is where a little education goes a long way. Just explaining that homeopathic potions don’t have any actual active ingredients (that haven’t been diluted out of existence) is often enough.
And of course we continue to play watchdog on alternative medical professions, such as chiropractic and naturopathy, who have managed to establish medicine without science, or with a very bad impersonation of science. They are great examples of what happens when you practice medicine without a clear and dedicated anchor to high quality science.
The usual suspects, of snake oil peddlers, magic energy devices, science denial, anti-vaxxers, COVID and other conspiracy theorists, are all still out there menacing the public health. Along the way we take some time to celebrate true breakthroughs in science and medicine, making a nice contrast to the perpetual false claims and promises of pseudoscience in medicine. And we take a look at our institutions and society as a whole, where are things going wrong and how can be get back on track.
We never expect victory. This is a perpetual struggle. Science, critical thinking, and professionalism are all extremely high energy states. They take constant work and introspection, and need institutional support. Meanwhile, there are many forces at work against them. There are psychological forces, such as the allure of false hope, the desire for cures that may not exist, or for easy answers to complex questions. There is the greed that drives a multi-billion dollar (at least) industry of fake medicine. And there are the inherent advantages that misinformation and emotional appeals have in mass and social media.
The modern world is challenging the resilience of all of our institutions, and science and medicine are no different. Hopefully our contributions are helping in some small way, through education and advocacy. But there is always more to be done.
Finally, thanks to all our regular readers and commenters and of course our patrons. You are a community of thoughtful and curious individuals interested in what we have to say about science and medicine. You are the vibrant community that keeps our humble blog afloat. This is especially true for our patrons. Thank you, and I look forward to more SBM in 2024.

So Long 2025
Steven Novella on December 17, 2025
Link

This will be my last post of the year and as usual I like to look back at how SBM is doing. We have now completed 17 years as a blog, which I think is something in itself. As always I have to thank David Gorski and all of the regular contributors here for their tireless work without any compensation except fulfilling the desire to make the world a slightly better and more rational place.
2025 has been a rough year, I think unequivocally the worst year we have had since starting SBM. At the heart of what we do is a philosophical conflict – Science-Based Medicine is a particular philosophy of medicine. We promote the idea that science, in all its aspects, is the best tool for determining which treatments are safe and effective, that safety and efficacy matter (yes, this has to be stated explicitly), and that an SBM approach to clinical decision making leads to the best outcomes. But it’s not enough to simply defend this approach, because it is horrifically complicated, so we also spend time delving into wonky discussions of how to optimize the SBM approach, knowing that this is an endless project.
On the other side (I will honestly try to be as charitable as I can here, but all the caveats apply about listening to hostile sources, which I definitely am), there are those who I think sincerely believe that science is not the ultimate determination of the utility of medical and health interventions. Or they may think their approach is scientific, but they are clearly doing it wrong, and for a variety of reasons. Often they are just stuck in a narrative that does not align with reality.
To take a recent example which I think illustrates this well, Mark Crislip just wrote about the current administration’s approach to pandemic preparedness, which focuses on making people healthy to be better able to resist infection. This is a typical wellness narrative, but Mark, who is an infectious disease expert, details that reality is quite different. Organisms evolved to be good at infecting us, and our immune systems are variable, better able to tackle some infections than others. Just relying on good nutrition is folly, it is subjugating science, evidence, and logic to an ideology.
But even mainstream professionals may sometimes go astray because they fail to consider the full spectrum of evidence. They fall for the “evidence-based medicine” fallacy – the notion that you can focus entirely on clinical trials without consider overall scientific plausibility. Or they are simply insufficiently educated about things like P-hacking, publication bias, the decline effect, or Bayesian statistics.
At the other end of the spectrum, however, there are con-artists, charlatans, and snake-oil peddlers. Of course they are going to infiltrate a movement that is premised on loosening the regulations and science-based standards of a multi-billion dollar industry. The wellness / alternative medicine industry is practically manufactured for deception and exploitation. It is a haven for every kind of fraudster. It is so intrinsic that it is often impossible to tell to what degree someone is a sincerely believer or a sinister con artist. I suspect most people in the wellness industry are a combination, somewhere along a continuous spectrum. The likely believe their own hype, but also have some sense that they are cutting corners and are just being “good salespeople”.
Then we throw into the mix the conspiracy theorists and fantasists, people whose relationship with reality is tenuous at best, or they simply have a critical failure in their reality-testing circuitry. There is a spectrum here as well. Neurotypical or average people have a tendency to fall for conspiracy theories, especially if they align with their narratives or world view. The wellness industry embraces the full spectrum, because it is an extremely useful tool for selling their wares. “Our magical claims are better than mainstream medicine, because you can’t trust mainstream medicine, because it’s all a giant conspiracy.” If you want to see how pervasive this narrative is, just browse TikTok for a bit (until your brain melts).
So why has 2025 been so bad? Even casual readers here know, as evidenced by our near obsession with a few people in the current administration. I have honestly had to make a concerted effort to avoid writing about RFK Jr., Dr. Makary, Vinay Prasad, and Jay Bhattacharya (my colleagues here have it covered pretty well), in order to have some balance here and continue to tackle some of the more classic topics at SBM. But every week there is some outrage that deserves discussion on SBM. It’s hard to avoid.
We are facing nothing short of the barbarians being inside the gates – the charlatans are in charge of health care at the federal level. RFK Jr. represents much of the very philosophy we are pushing back against – he is not an expert but thinks he knows better, he is a conspiracy theorist, and he buys into all the dubious narratives of the wellness industry. He is blatantly anti-vaccine, despite his coy protestations.
Under his tenure healthcare regulation in the Trump administration has been eviscerated. He is waging war against vaccines, and has gone farther than even we feared. Pandemic preparedness is a joke, and we all have to hope that we are not hit with the next pandemic anytime soon. The CDC, FDA, NIH are all compromised. We can no longer trust their recommendations or rely upon their expertise.
The “Make America Healthy Again” (MAHA) movement is a dream of the wellness and alternative medicine industries. It is a fantasy that flies in the face of a century of medical science and evidence, it is a triumph of narrative over reality. Sure, like much of CAM, there is some basic lifestyle advice thrown in – we didn’t need MAHA to tell us to eat right and exercise regularly. Otherwise they are focusing on the wrong things – we will not make America healthier by banning food dye and reducing access to vaccines.
RFK also makes it hard not to be cynical. He is obsessed with environmental contributions to poor health. Of course, they are real, but he grossly overestimates their role in health and downplays may other legitimate factors. And yet, on the one environmental issue that perhaps has the biggest actual effect on health – pollution from fossil fuels – RFK instantly caved to Trump’s global warming denial and fawning support for the fossil fuel industry. On his second day as HHS secretary he cut NIH programs studying the effects of pollution and climate change on public health. He has pivoted away from his environmentalist roots and refuses to push back against Trump’s climate change denial.
It is scary to think that we potentially have three more years of people like RFK dismantling the institutions of medicine. They weren’t perfect to begin with – medicine is hard, and regulations are complex, but at least there was a steady push for incremental improvements. Now a crank conspiracy theorist is doing his best to tear it all down.
We will get through this, and hopefully the public will gain some insight into what happens when you abandon objective standards for feel-good snake oil. But it remains to be seen how much long term and even permanent damage will be done in the meantime. There will be a death toll attached to RFK’s nonsense (as there already has been), we just don’t know how many zeroes there will be.
Either way, SBM will be here in 2026 and hopefully beyond to tell you all about it.

Evidence-based medicine vs. basic science in medical school
Last week Dr. Vinay Prasad wrote a Substack arguing that medical students should learn the principles of evidence-based medicine before basic science.This is a recipe for amplifying the main flaw in EBM that science-based medicine was meant to correct, and Dr. Prasad’s arguments would have been right at home on an integrative medicine blog. [Note ADDENDUM.]
David Gorski on May 22, 2023
Link

Since we hit the 15-year anniversary of this blog, I’ve been thinking more about the overall concept of science-based medicine (SBM) as compared to the evidence-based medicine (EBM) paradigm than I probably have since Steve Novella first invited me to join this blog’s founders in 2007 as he was planning on launching the blog. I remember some epic email exchanges that took place during the weeks leading up to the blog’s launch on January 1, 2008. It is in this light that I will address a proposal made on Friday by Dr. Vinay Prasad and explain why I think it’s a very bad idea—not just a bad idea but a nice jumping-off point to address how EBM fundamentalism has contributed to two major problems in medicine, that of the “integration” of quackery as though it were medicine and COVID-19 minimization and even outright antivaccine attacks on COVID-19 vaccines. The title will likely be a tip-off: Rethinking Medical Education: Evidence First, Biology Later. In the post, Dr. Prasad proposes “placing evidence-based medicine at the fore, right from the get-go,” by teaching EBM first and basic biological sciences after that.
[ADDENDUM: Dr. Prasad now says that his essay was written using Chat GPT. Comments at the end of the post. Let’s just say that it’s not the flex that he thinks it is.]
While one can look at this proposal as just another iteration in the eternal debate over the relative roles of basic and clinical science education in medical school, I look at it a bit differently. Specifically, I look at it through the lens of the criticisms that we at SBM have been making about EBM since the very beginning and, more recently, have been expanding to cover observations that I’ve noticed regarding the tendency of prominent EBM advocates (e.g., John Ioannidis) to embrace questionable ideas about COVID-19 and public health.
SBM vs. EBM
To set the stage, let me briefly quote from Steve Novella’s inaugural post announcing the blog, after which I will begin addressing the problems with Dr. Prasad’s idea and how it fits into the problem with EBM that we at SBM have been pointing out for over 15 years:
Within the practice of medicine there is already a recognition of the need to raise the standards of evidence and the availability of the best evidence to the practitioner and the consumer – formalized in the movement known as evidence-based medicine (EBM). EBM is a vital and positive influence on the practice of medicine, but it has its limitations. Most relevant to this blog is the focus on clinical trial results to the exclusion of scientific plausibility. The focus on trial results (which, in the EBM lexicon, is what is meant by “evidence”) has its utility, but fails to properly deal with medical modalities that lie outside the scientific paradigm, or for which the scientific plausibility ranges from very little to nonexistent.
All of science describes the same reality, and therefore it must (if it is functioning properly) all be mutually compatible. Collectively, science builds one cumulative model of the natural world. This means we can make rational judgments about what is likely to be true based upon what is already well established. This does not necessarily equate to rejecting new ideas out-of-hand, but rather to adjusting the threshold of evidence required to establish a new claim based upon the prior scientific plausibility of the new claim. Failure to do so leads to conclusions and recommendations that are not reliable, and therefore medical practices that are not reliably safe and effective.
This is why the authors of this blog strongly advocate for science-based medicine – the use of the best scientific evidence available, in the light of our cumulative scientific knowledge from all relevant disciplines, in evaluating health claims, practices, and products.
Let me start out by pointing out a commonality that we share with Dr. Prasad, for all our past criticism, which is that the standard of evidence in medicine does need to be raised. Indeed, before the pandemic I once argued that the FDA is too lax in its standards for drug and device approval, rather than too strict, as many libertarians who want to weaken the FDA like to argue. Where we differ is that, while SBM argues for the use of all the best scientific evidence available from all disciplines, Dr. Prasad, like many of what I have come to call “EBM fundamentalists” thinks that the EBM paradigm is the only form of evidence that matters, a phenomenon that I have long been calling “methodolatry,” a term that I discovered in 2009 thanks to a senior epidemiologist and defined as the profane worship of the randomized controlled trial as the only valid form of clinical investigation. Moreover, my discovery of methodolatry was in the context of an earlier pandemic, namely the H1N1 influenza pandemic of 2009-2010 and how EBM fundamentalists did something very similar to what they have been doing during the current COVID-19 pandemic, demanding RCTs for public health interventions for which RCTs are either impractical, too slow, or even unethical.
In 2007, the main concern that had led to the proposal of SBM was the rise of what was then commonly called “complementary and alternative medicine” (CAM) but has since been consciously rebranded by its proponents as “integrative medicine” or “integrative health” in order to remove the word “alternative” (which has a bad connotation) and “complementary” (which implies a lower status than EBM). The basic idea behind integrative medicine is to “integrate” what advocates sometimes call the “best of both worlds”; i.e., to “integrate” various alternative medicine modalities into conventional evidence-based medicine.
Believe it or not, at the time I wasn’t entirely on-board with the whole concept of SBM and had to be convinced, hence the epic email exchanges. (This was 2007.) The reason was that I could not accept the central premise of SBM, specifically that the EBM paradigm hugely undervalues basic science considerations in considering the evidence for various highly implausible treatments being “integrated” into medicine and that Bayesian prior plausibility should be introduced into EBM to correct that “blind spot.” If you look at the EBM pyramid, you will see that basic science is near the bottom, while RCTs and meta-analyses of RCTs are at the very top.
Let’s just say that I had to be convinced of SBM’s central premise, and it took a lot of convincing to accept that my view of EBM at the time was hopelessly naive in just assuming that surely, EBM pyramid aside, basic science must be a major consideration and that RCTs of “impossible” treatments like homeopathy should not be done.We used to routinely note that basic science considerations are relegated to the very bottom of the pyramid, as “level 5 evidence” consisting of “expert opinion without explicit critical appraisal, or based on physiology, bench research or ‘first principles.'”” This is a model that fails when it comes to addressing alternative medicine treatment modalities that hundreds of years of chemistry, physics, and biology show to be impossible from a basic science standpoint, our favorite example being homeopathy.
Given that I recently discussed this concept in detail, I won’t rehash it (much). Instead I will point to a classic SBM post by Dr. Kimball Atwood asking: Does EBM Undervalue Basic Science and Overvalue RCTs? (Hint: the answer is yes.) He also wondered whether it is a good idea to test highly implausible health claims. (Hint: the answer is usually no, at least not using RCT efficacy trials of the sort placed near the very top of the EBM pyramid of evidence, just under systemic reviews, evidence syntheses, and meta-analyses of RCTs.)
As I like to ask: Which of the following is more likely, that a 30C homeopathic solution of…something…that has been diluted on the order of 1037-fold more than Avogadro’s number and thus is incredibly unlikely to contain even a single molecule of that something has a therapeutic effect or that the RCTs concluding that it does reveal the problems and biases in clinical trials? As I also like to say, given the usual p-value of 0.05 designated for “statistically significant” findings, under ideal circumstances, with perfectly designed and executed RCTs, by random chance alone 5% of these RCTs will be “positive.” Of course, in the real world, RCTs are not perfect, either in design or execution, and the number of “false positives” is therefore likely considerably higher than 5%. Yet, basic science alone tells us that a 30C homeopathic remedy is indistinguishable from the water used to dilute it, which means a placebo-controlled RCT is testing placebo versus placebo and “positive” results show us nothing more than the noise inherent in doing RCTs.
Now, as I did then, I will also add this caution: EBM is correct that “first principles” in basic science cannot tell you whether a treatment works or not—except for scientifically impossible treatment modalities like homeopathy, that is. Aside from testing treatments that violate multiple well-established laws of physics, RCTs are indeed necessary to determine if a treatment is safe and effective. It is also true that history is littered with treatments that appeared effective in preclinical in vitro and animal models but failed to pan out in clinical trials, a point that Dr. Prasad alludes to, as you will see, but using an argument that can backfire. However, in contrast, if very well-established basic science tells you that a treatment modality is impossible and cannot work, that (like homeopathy) and would require that multiple well-established laws and theories of chemistry and physics be not just wrong, but spectacularly wrong, in order for it to “work,” then that should be enough. RCTs should not be necessary. Indeed, they would arguably be unethical, because one of the major ethical requirements for clinical trials is that the treatment being tested be grounded in good science based on what is known at the time of the trial, and clinical trials of many alternative medicine modalities (e.g., homeopathy, reflexology, reiki, and more) violate this precept.
Contrary to straw man attacks against SBM, we only ever intended the idea of considering prior plausibility as a way of ruling out impossible treatments like homeopathy without RCTs, which inevitably produce false positives that advocates of quackery like to cite, and to put RCTs of highly implausible but not necessarily totally impossible treatments (e.g., acupuncture) into context by assessing how likely it is that a positive RCT represents a “true positive.” (Hint: very unlikely.) As we have long argued, medicine does need more science, but unlike what EBM fundamentalists seem to believe, RCTs are not the be-all and end-all of clinical science, particularly for nonpharmaceutical and non-device interventions used in public health, for which RCTs are not infrequently incredibly impractical, expensive, or even—in the middle of a pandemic, for instance—unethical.
So what does this have to do with Dr. Prasad’s proposal?
EBM (and RCTs) über alles
Let’s now see what, exactly, it is that Dr. Prasad proposes and how he justifies it:
Medical education has got it backward. We’re front-loading biology and anatomy, then backfilling with the principles of evidence-based medicine. We need to flip this equation, placing evidence-based medicine at the fore, right from the get-go. Patients don’t care about the biological mechanisms; they care about what helps them get better, regardless of the underlying science. That’s the crux of our argument today.
Let’s take a step back and look at the evolution of medical education over the 20th century. With the boom in biological sciences, our understanding of human anatomy, physiology, and pathology has expanded exponentially. This knowledge has undoubtedly been invaluable, forming the bedrock of our modern approach to diagnosing and treating diseases. There’s a problem, though. By placing biology at the center of medical education, we’ve inadvertently created a situation where we’re missing the forest for the trees.
“Patients don’t care about medical mechanisms, they care about what helps them get better, regardless of the underlying science”? Andrew Weil or David Katz couldn’t have said it better himself. That statement definitely sent up a huge red flag! After all, how many times have critics of SBM said something very similar over the years? In fairness, it is true that patients don’t care so much about the basic science compared to the efficacy and safety of a treatment, but in determining which treatments are likely to work basic science does and should have a major role. We physicians are not patients. We do have to care about basic science. Some of us—such as myself—are even also basic or translational scientists. We’ve been arguing for well over a century, ever since the Flexner report, over how much basic science we as physicians need to know, but the answer has always been that basic science is as important as clinical knowledge in medical education.
Let’s get to the part of Dr. Prasad’s argument that made my jaw drop:
Simultaneously, another revolution was brewing in the medical field during the 20th century: the rise of evidence-based medicine. Randomized clinical trials (RCTs) became the gold standard, the north star guiding our therapeutic decisions. The power of RCTs lies in their objectivity, their ability to tell us, unequivocally, whether an intervention works or not. It’s not about hypotheses or educated guesses; it’s about cold, hard data. Yet, this pillar of modern medicine often plays second fiddle in our educational curriculum.
Here’s the rub: when we instill a deep-seated reverence for biology before introducing the principles of evidence-based medicine, we’re setting up our future doctors for cognitive dissonance. It’s harder to accept that a treatment doesn’t work when it “should” according to biological principles. As a result, evidence that contradicts our understanding of biology is often met with skepticism, even outright rejection. This is a disservice to our patients, who ultimately care about outcomes, not biological plausibility.
This is EBM fundamentalism at its most blatant. Notice how Dr. Prasad equates EBM with RCTs. He doesn’t mention anything else, including epidemiology, which, if you look again at the EBM pyramid above, you will notice to be under RCTs—under nonrandomized clinical trials, actually. When I refer to “methodolatry,” this is exactly what I’m talking about, the assumption, sometimes stated but often unstated, that the RCT is the only valid method of clinical investigation that can produce a definitive answer to a question. My favorite counterexamples to this are both from epidemiology. The first is how it is by epidemiology alone that we discovered that smoking causes lung cancer—and so many other diseases. The second is how it is primarily though epidemiology (along with a few RCTs, admittedly) that we know that vaccines are not associated with an increased risk of autism. It would be completely unethical (and probably impossible) to attempt RCTs addressing the questions of whether smoking causes cancer and vaccines cause autism now. In a more balanced paradigm, we in medicine would use the right scientific tools to address the correct question. RCTs are undoubtedly the right scientific tools for the initial testing of whether a new drug, surgical procedure, or medical device is safe and effective for its intended purpose. After that, it gets more…complicated, particularly when ethical considerations of clinical equipoise start entering into the equation.
Also, RCTs are “objective”? Yes, but only in comparison to other methods and only along a continuum of objectivity. Like any other scientific tool, no RCT is truly completely objective; the idea is to approach objectivity, to minimize bias as much as is feasible and ethical. At their best, RCTs can and do tell us unequivocally whether an intervention works or not and are generally the least bias-prone method of doing so. In an ideal world, they would all accomplish that. However, Dr. Prasad himself before the pandemic routinely used to demonstrate how they often do not. It turns out that RCTs are messier than this idealized version portrayed by Dr. Prasad would indicate. We’ve even discussed this at SBM and elsewhere not just in terms of RCTs of “integrative” medicine interventions but of just…interventions. At times, we’ve even praised Dr. Prasad for his previous work. Unfortunately that was in 2017, and it’s 2023 now. Dr. Prasad has changed.
The “rub” that Dr. Prasad cites is also a red flag. In essence, he is arguing against the very concept of SBM in that he seems to view considering scientific prior plausibility to be a bad thing. Acupuncturists and homeopaths couldn’t have said it better! Let me just flip Dr. Prasad’s assertion that it is “harder to accept that a treatment doesn’t work when it ‘should’ according to biological principles” on its head to show you what I mean: It is easier to dismiss scientific considerations that assess the prior plausibility of an intervention, the prior probability that it will work, as very low or even impossible if basic science considerations don’t matter. In addition, how, precisely, do we decide which clinical interventions to test in Dr. Prasad’s vaunted RCTs if we don’t have some basis upon which to assess the likelihood that they will work? His version of EBM appears to reject completely the current drug and device development paradigm, which starts with basic science interventions, moves to testing the interventions in cell culture and animal models, and then to small scale pilot clinical trials, and finally to the large RCTs that Dr. Prasad presents as the best method to test clinical interventions. Under a paradigm like the one described above, one could justify testing almost anything just on the basis of anecdotal evidence, which is the lowest form of evidence in clinical medicine, including homeopathy. After all, patients (and apparently Dr. Prasad) don’t care about basic science, just efficacy and safety, so why not?
Don’t believe me? Dr. Prasad states it outright near the end:
This brings us to the crux of our argument: the need to revamp the medical curriculum. Let’s start with the history of medicine, the triumphs, and the pitfalls. Next, introduce the principles of clinical trials, the linchpin of evidence-based medicine. Show students examples of medical reversals, such as the Cardiac Arrhythmia Suppression Trial (CAST), where our understanding of biology was turned on its head by clinical trial data. Such an approach will underscore the importance of evidence over theory, preparing our future doctors to prioritize patient outcomes over biological dogma.
First, beware of anyone who uses phrases like “biological dogma.” I can’t count the number of times I’ve seen, for example, homeopaths use this or similar terminology to dismiss basic science considerations that show that homeopathy is scientifically impossible. In fairness, I will also state that it would not be a bad idea to teach a lot more about the history of medicine, in particular of ideas widely accepted that turned out to be wrong, but one does have to remember that there is only a limited amount of time. Adding topics to the curriculum nearly always results in deemphasizing or eliminating other topics. Which topics would Dr. Prasad deemphasize or eliminate?
It is, of course, true that the CAST trial did produce a result in conflict with the scientific understanding of cardiac physiology at the time in which the intervention to decrease arrhythmias after a myocardial infarction (heart attack) unexpectedly resulted in increased mortality. It is difficult not to note, however, that Dr. Prasad doesn’t explain how his approach of dismissing or downplaying biology would have avoided the need to carry out this particular trial given what was known at the time about arrhythmias after heart attacks or how he would have discovered that the routine use of these drugs (class I antiarrhythmic drugs) might increase mortality after acute myocardial infarction without the clinical trial. So why did the trial fail? The reason turned out to be that in the setting of myocardial infection these drugs, rather than decreasing arrhythmias, actually generate arrthymias, some of which can be fatal,. Moreover, contrary to what Dr. Prasad argues, a counterargument can be made that the reason this trial failed because of inadequate consideration of preclinical and basic science suggesting that perhaps these drugs could generate arrhythmias in the context of a myocardial infarction, not adherence to “biological dogma.”
Here’s what I mean, starting with the abstract of a 1994 paper on the trial:
Although active treatment in CAST-I was associated with greater mortality than placebo with respect to almost all baseline variables, the therapeutic hazard was more than expected in patients with non-Q-wave myocardial infarction and (for total mortality) frequent premature VPDs and higher heart rates, suggesting that the adverse effect of encainide or flecainide therapy is greater when ischemic and electrical instability are present. The relative hazard of therapy with moricizine in the sicker CAST-II population was greater in those using diuretics. Thus, although these drugs have the common ability to suppress ventricular ectopy after myocardial infarction, their detrimental effects on survival may be mediated by different mechanisms in different populations, emphasizing the complex, poorly understood hazards associated with antiarrhythmic drug treatment.
Later in the paper, it is noted that in animal models of cardiac ischemia (low blood or blocked blood flow), these drugs increased arrhythmias and death in the experimental animals who received them. I’m not a cardiologist or an expert in cardiac ischemia, but one wonders if the results of these animal studies might have predicted the problem seen in this trial. I don’t claim that they did, but to hold up this trial as an example of a clinical trial whose results radically changed our understanding of biology strikes me as reaching a whole heck of a lot. Maybe my take is incorrect, but I don’t think I’m incorrect to suggest that this is not a great example of our understanding of biology being totally changed by the results of a clinical trial; rather, it sounds like more of a refinement. Cardiologists and/or cardiac physiologists who might be reading can correct me if I’m wrong.
None of this justifies Dr. Prasad’s broad claim:
In essence, we’re arguing for a reordering of priorities in medical education. While the knowledge of biology and anatomy is crucial, it should not overshadow the importance of evidence-based medicine. By introducing students to the principles of evidence-based medicine early on, we can create a generation of doctors who put patient outcomes first, regardless of biological plausibility. It’s not a case of one over the other; it’s about getting the sequence right.
No, clearly to Dr. Prasad it is a case of one over the other, with EBM being more important, or “getting the sequence right,” so much so that Dr. Prasad wants it taught first. Also Dr. Prasad advocates EBM “first, biology and anatomy later,” but how can a student actually approach EBM claims without a knowledge of biology and anatomy? For a simple example, how would one approach a knowledge of the EBM basis for surgical procedures for gallstones and cholecystitis without a knowledge of gallbladder and biliary tree anatomy? How would one approach the EBM basis of chemotherapy for various cancers—or pick just one!—absent a basic science understanding of the molecular basis of the specific cancer being considered and the biological mechanisms of action of the drugs used?
One of Dr. Prasad’s commenters put it well:
I have to say I disagree with Vinay on this one. In order to understand the underlying basis of medicine, you have to know some basics physiology (including neuroscience), anatomy and biochemistry. That provides one with the tools to think critically which is so crucial when it comes to evidence-based medicine and RCTs. That’s especially so when not all RCTs are created equal or done correctly (as indeed Vinay continually points out when it comes to cancer drugs).
Personally, as an advocate of SBM, I tend to agree with this commenter, with one exception, namely that there is no “may” about it:
Vinay may be setting up a false dichotomy. Doctors need to be trained on how to integrate and coordinate their knowledge of biology, pathology, anatomy, etc with their knowledge gained from reading reports of RCTs.
This is almost a statement of what SBM is about! He’s so close. I would say that doctors should be able to integrate their knowledge of basic and preclinical science with RCT results in order to assess the probability that a given RCT has produced an accurate reflection of the efficacy and safety of an intervention—in other words, Bayesian reasoning that takes into account not just RCTs.
Methodolatry versus SBM
Dr. Prasad’s “modest proposal” is at its heart methodolatry writ large. To him EBM is RCTs. (If Dr. Prasad accepts that EBM is and should be more than just RCTs, he gives no hint of it.) Basic science doesn’t matter much, his protestations otherwise that it is just as important (but should be taught only after EBM) notwithstanding. Indeed, it’s hard not to look at his entire proposal as a way to indoctrinate students into a version of EBM that is at its heart methodolatry and then “backfill” (as he puts it) this narrative with basic science that supports it. Whether he realizes it or not, Dr. Prasad is supporting a model that has long been used to misrepresent quackery as potentially effective and put it on steroids. I suppose that I shouldn’t be surprised, as this is a doctor who has long dismissed debunking of quackery and antivax misinformation as a waste of time. Unfortunately, now he has become a poster child for why this debunking is so difficult and necessary, even as he himself complains about criticism of his many dubious pronouncements about the pandemic.
I do think that more training in clinical trial methodology in medical school would be very helpful. Truth be told, three decades ago there wasn’t very much training of this sort in my medical school curriculum, although I am told that there is more now. The problem is that, contrary to what methodolatrists like Dr. Prasad argue, it is impossible to contextualize RCT findings without a solid grounding in basic anatomy, physiology, and biochemistry. Rather than separate the two, I would argue that medical education need more just plain science, including basic and clinical science, plus much more critical thinking. Dr. Prasad, for all his critical thinking about oncology trials in the past, shows that his critical thinking is very selective, and a physician’s clinical thinking about medicine and RCT results should be much broader—as well as based in science.
ADDENDUM May 23, 2023: Yesterday, Dr. Prasad wrote a post for his Substack stating that his entire post had been written by Chat GPT, the AI chatbot that’s been in the news so much lately:
I am actually writing this column. But I did not write a column that appeared last Friday, entitled Rethinking Medical Education. That column was 100% written by Chat GPT.
It all got started with Timothee Olivier. He had a wild idea. He asked Chat GPT to write a column in my style. (Below is his verbatim prompt). He suggested I post it to see the response. He made no edits to it. The column I posted was that.
When I read it the first time, it sounded a lot like something I would say. Because I said something similar 10 years ago. That article was published in academic medicine. Of course, Timothee knows my views on this topic, so he gave some guidance. At the same time, the essay that Chat GPT wrote is different than what I would write. It is drastically different than my 2012 essay.
I have to admit that while reading it, although I thought it was one of Dr. Prasad’s lesser efforts, from both the standpoint of style and argument, it didn’t occur to me that he hadn’t written it. No doubt Dr. Prasad, if he’s seen my post at all, is chuckling at putting one over on me. However, before you think I screwed up, just look at this prompt that was used, as described in his “revelation”:
Hello, ChatGPT, could you write a 1000 words essay, in the style of Vinay Prasad, MD- MPH, writing in the style of his personal substack, punchy as usual. The whole idea of the essay would be that medical training should be shorter, but focused on evidence-based medicine and patient’s care very early on. One paragraph would be about the fact that because biology made great advances during the 20th century, it became central in medical education. The second paragraph should emphasize that the other main pillar of medicine that occured in the 20th century is evidence-based medicine, with randomized clinical trials at its core. Third, we should underline that timing in the curriculum matters : if students are first tought about the primacy of biology, they will later reject evidence-based result based on implausibly biological findings, yet what matters to patients is what is beneficial to them, whatever the biological reason. In a last paragraph, Vinay would argue that the medical curriculum should start with history of medicine, great principles of trials, examples of medical reversal, when biology was later reverse in randomized clinical trials like in CAST, and that medical eduction should stand first on evidence-based principle before teaching basic principles of biology and anatomy. Make more paragraphs than less if necessary. Avoid the term “paradigm shift”. The style has to be easily understandable for lay people also. The first sentences should be short and clear about the whole argument of the essay.
So Olivier and Prasad basically told Chat GPT what to write and how to write it, using a prompt that is, at just under 250 words, one quarter of the word count of the final essay. In fact, the Chat GPT prompt is so long and detailed, that Dr. Prasad might as well have written the damned post himself, as it probably took as much work to come up with that prompt as it would have just to write the post. It also makes it very clear that, even if the actual essay was written by Chat GPT, it is basically Dr. Prasad’s argument. His prompt tells you that it was, as he and Olivier basically laid down very clear instructions for what the chatbot should write and even some phraseology that should be included. I also assume that he wouldn’t have posted it if he had major problems with it. So, ChatGPT or no ChatGPT, this is clearly nothing more than one of Dr. Prasad’s bad arguments.
I’ll also note that I’ve tried the same sort of experiment with Chat GPT and been very much less impressed with the results. (It’s not as though there isn’t a copious amount of verbiage for the chatbot to train on to mimic my style, either.) For instance, I once asked it to write a post in my style about why homeopathy is impossible, and it sounded nothing like me. Maybe I should try again.

“How do you feel about Evidence-Based Medicine?”
Harriet Hall on May 29, 2012
Link
That was the question asked on a Medscape Connect discussion
I did a double-take. How do you feel? Could anybody object to the idea of basing treatments on evidence? The doctor who started the discussion asked:
Besides using EBM, a lot of my prescribing comes from anecdotal experience and intuition. How about you? Where do you get your information from that you use to treat your patients? Do you always ascribe to EBM, or do you deviate from it with certain medical conditions/patients?
I had naively thought that my profession uniformly embraced EBM. How could they not? The commenters broke my bubble big-time. Some of them summarily reject EBM… although it appears that what they are rejecting is not what I understand EBM to mean.
Note: I have excerpted the following quotations from the website, preserving the errors.
Some typical comments
    • EBM is touted by ivory-tower bozos
    • EBM is a pile of data-computer generated. You do not know which data are true or false.
    • The science that we push has money attached to it.
    • only 15% of what we do has any evidence base.
    • I would like EBM better if it actually worked
    • EBM: the biggest smoke screen in medicine
    • Much of the “evidence” today is fabricated and doctored by Big Pharma
    • hard working physicians are screwed from all directions even from statisticians who sit on rear end all day talking to computers and have all the time to pontificate without sweating looking after a dying patient or taking care of a bleeder that wont quit!
    • Find a paper that says one thing, and I will find another that refutes
    • Experience trumps EBM
    • I see EBM as a way of “dumbing down” medicine to allow NP’s to try to practice medicine
    • Allopathic medicine doesn’t have a monopoly on knowledge
    • EBM is code for rationing and cookbook medicine such that experience doesn’t matter.
    • it drives physicians to treat the diagnosis instead of the patient (the underlying flawed premise of EBM)
    • In this system, how are we any different than a really smart computer
    • It’s become the new religion.
    • EBMs come from Committees.
    • EBM is a “cookbook”. One size fits all. Experience and years of training does not matter
    • With regard to meningitis vacccine, now that we are vaccinating everyone, it is less likely that people will look for a better way to prevent full blown meningococcemia
    • EBM is statistics. Nothing else
    • I use EBM as a guideline, but sometimes they fly in the face of my own experiences. As the punchline in an old joke, “Are you going to believe me or your lying eyes?” There is validity in experience.
    • THE INSURANCE COMPANIES PUSHED “EVIDENCE BASED MEDICINE” ON US!
    • there is actually NO EVIDENCE that a practice based on so called EMB is effective or even relevant to real patients.
    • I just love it when I get a note from an RN questioning my prescription and citing “Evidence Based Medicine”
    • Intuition based medicine, like art, is always about a century ahead of science.
    • “Evidence based medicine” is a form of doublespeak. It restricts recommendations to those proven in clinical trials, which don’t exist for most clinical situations
    • EVERY ten years the Literature changes!!! EBM today is the quackery of tomorrow
    • As long as drug companies own the experts and fund the vast majority of studies AND have the right to publish the findings or not as they see fit we will NEVER have fully reliable evidence… therefore evidence based medicine might well be WRONG medicine.
Keep in mind that these are not clueless celebrities, anti-vax mommies, or crank CAMsters. These are mainstream medical doctors. I weep!
Some of their less irrational objections (with my comments)
    • Evidence from a trial with selected patients doesn’t necessarily apply to individuals in your office. (True. That’s where judgment and common sense come in. The trick is to not blindly follow anything labeled “evidence” but to consider all applicable factors and apply available evidence rationally.)
    • Early adoption of EB treatments superseded by new data. (New data… isn’t that a good thing? Meanwhile, we can’t just do nothing while we are waiting for better data.)
    • EBM gives us a cookbook. No progress was ever made by following a cookbook. (EBM was never intended to be a cookbook. We don’t follow a recipe, but apply available knowledge to individual patients.)
    • I am not enthralled with EBM used by committee/hospital edict. (Me either. Who loves committees? That’s not the way it’s supposed to work).
    • population studies with correlations are helpful but can never substitute for scientific, logical reasoning applied to the individual patient in question. (And they were never intended to substitute; they are intended to inform.)
    • we’re supposed to turn in our fellow physicians who over-test or over-prescribe based upon “Evidence based Guidelines” (EBM is not Big Brother. Is this just paranoia, or have physicians actually been “turned in” or disciplined for not following these guidelines?)
    • what I fear is rigidly taking the conclusions of a research study, … and FORCING physicians to do things that way. (I fear that too. It shouldn’t happen.)
There were a few more rational comments:
    • I cannot comprehend what the alternative to EBM would be.
    • It’s really just a label for “good medicine”
    • there can be no doubt that our individual clinical experience is biased.
    • we are all practicing EMB all the time but are being fed a brand of EMB with which we are not always comfortable (SBM is not comfortable with the current brand of EBM either).
    • Are we reverting to the Dark Ages?
    • EBM should be used as a tool in the decision making process, but should not be deferred to in place of independent thinking nor considered a gold standard not to be deviated from.
    • there is NO better alternative.
    • EBM is a well-meaning and useful concept if tempered with judgment and judicious application. My challenge with it is that it has been turned into a religion, and we physicians are supposed to worship at its alter without the addition of any critical thinking.
    • Evidence medicine implies that the studies have been peer reviewed and meet rigid criteria regarding study design and conclusions. It should not mean that small, poorly designed studies with unsupported conclusions are cited as part of EBM recommendation.
    • simply saying “I know what I know” and practicing anecdotal medicine by it self is simply dangerous
One commenter provided a link to SBM and called us “the militant wing of the EBM movement.” Kind of ironic when we established the blog as a protest against the flaws in EBM.
Conclusion
Most people would agree that medicine should be based on evidence and on science whenever possible. Good medical practice is not an “art” but an applied science that takes all available knowledge into consideration and applies it to the individual patient using careful judgment, compassion, and common sense. EBM is a worthy goal; but as currently implemented, it has fallen short of its potential. We on SBM see it as neglecting basic science, prior plausibility, and common sense. Others apparently see it as a dogmatic institutionalized draconian interference with practitioner autonomy. Incredibly, some scientifically educated medical school graduates still see evidence as inferior to intuition and experience: those are the ones who give me chills.

An SBM Advocate Goes To Washington
An update on the strategies and deceptions used by the CAM movement to promote their treatments.
Steven Novella on April 5, 2023
Link

Last week I was invited to be involved in a roundtable discussion for a US senate committee on healthcare. This was a closed session without reporters, so I don’t feel at liberty to discuss details, but it did help crystalize for me the current thoughts and strategies of those promoting complementary and alternative medicine (CAM). I can give my overall impressions of where I think we are, partly informed by my recent experience.
Alternative, No – Complementary, No – Integrative
CAM proponents use a lot of bait-and-switch or what David Gorski has dubbed “Trojan Horse” strategies to push their treatments. Interestingly, it’s mostly the same exact treatments that have been around for 50 years or more. When we talk of CAM we are still mostly referring to homeopathy, acupuncture, chiropractic manipulation, energy medicine, and herbal medicine. There are branding tweaks, or new versions of these things, but the core concepts are the same. It doesn’t matter if you call them alternative, complementary, or integrative. It’s the same stuff.
The evolution of the branding, however, is to reassure those who care about things such as the standard of care in medicine that they will not be replacing proven therapies, and are offered in addition to them. First, this is never entirely true. If a CAM method is being placed somewhere in the hierarchy of treatments, then it is displacing or delaying non-CAM therapies. Also, health care is often resource limited. If a patient is availing themselves of acupuncture, that is using up their limited resources of time and health-care dollars. Proper treatment trials should also be done one at a time, or else you can’t know what is working. So while you are trying acupuncture (or whatever) you may not be doing the next thing.
Further, the very concept of “integrative” medicine is flawed to its core. What, exactly, are they integrating into what? Try to answer this question without just referring to euphemisms – no alternative or complementary methods, traditional, non-mainstream, etc. None of these really work, nor can they be operationally defined. If, for example, you define it as integrating non-mainstream methods into mainstream medicine, that provokes the question of why they are non-mainstream in the first place. And also – there already is a mechanism for that. It’s called science and evidence. The implication is that we should be integrating methods based on something other than the best available science. And once they are integrated, wouldn’t that then make them mainstream? So then what are they?
All of this rhetorical dancing is to avoid the obvious answer – integrative medicine is all about creating a double standard (legally, academically, professionally) so that treatments and methods that are below the line of scientific medicine can be integrated with those that are above the line. That’s it, and the laws that are pushed to promote CAM explicitly carve out this double standard.
Integrative medicine is preventive medicine/health promotion
This is the most annoying aspect of the CAM propaganda – the retconning of CAM into preventive medicine. This is nothing short of gaslighting. The first piece of this strategy is to slander mainstream medicine by saying it is not about disease prevention, only disease intervention. This is not to say we do a perfect job sufficiently emphasizing prevention. Healthcare is complex, it is very difficult to get people to change their habits, and we do nothing perfectly. But this does not mean we don’t do prevention or that CAM providers are better at it.
Most primary care is about disease prevention and health promotion. This is all science based – good nutrition, sleep hygiene, exercise, taking aspirin for vascular prevention, treating high blood pressure, lowering cholesterol, smoking cessation, etc. The amount of research being done on preventive strategies and slowly being integrated into the standard of practice is staggering. And it’s all mainstream science-based medicine. None of the CAM modalities I listed above have any preventive value (or arguably, any value).
But this line is rhetorically very powerful. If they can slip in this equivalency, then they have already won. To further support this approach CAM proponents are very liberal in defining what is CAM. They would love to include all nutrition and exercise – but again, this is anti-historical gaslighting.
Part of this complete breakfast
I believe I was the first one here to use this reference, because it is perfect for what is happening. Remember the old commercials where something like a breakfast pastry was shown next to orange juice, eggs, fruit, and toast and said to be “part of this complete breakfast”. Even as a child I saw through this scam – sure, it may be part of that breakfast, but it’s an irrelevant part.
Good science-based medicine should endeavor to isolate variables as much as possible. That is what the entire placebo-controlled trial is about. We cannot make causal conclusion unless the variable of interest is isolated. The problem for CAM proponents is that when you properly isolate the variable that is at the core of their treatment, it doesn’t work. After thousands of clinical trials, for example, acupuncture researchers still have not been able to demonstrate scientifically that acupuncture points mean anything. They do not appear to exist – their own research concludes this. Similarly, there is no “life energy” behind energy medicine, subluxation theory has been essentially disproven, and the principles of homeopathy are demonstrable nonsense.
The ideas and claims at the heart of these CAM methods have never been scientifically demonstrated, and go against pretty much all of modern biology, physiology, biochemistry, and even sometimes the laws of physics. So what’s a CAM proponent to do? One approach is to simply never isolate these core variables in a clinical study. Mix in a bunch of interventions that are almost guaranteed to be helpful (tracking nutrition, health coaching, sleep hygiene, compliance monitoring) and then claim that the CAM treatment “worked”. Or simply, don’t properly blind the study. That way you can claim that non-specific placebo effects are evidence that the treatment works. One way to do this is pragmatic studies, looking at real-world use. But these studies cannot be used to make efficacy claims – for the reasons stated, they are not blinded and don’t isolate variables. Pragmatic studies should only be done on treatments that have already demonstrated efficacy.
Appeal to anecdote
I almost always hear the phrase, “I have seen it work in my practice” from CAM proponents justifying their non-science based interventions. Of course, this is anecdotal, it means close to nothing. Anecdotal evidence leads us to the conclusions we want to be true (quoting Barry Beyerstein), not the truth. Legions of patients swore by all kinds of nonsensical treatments throughout history, that we now know to be nothing but snake oil.
The appeal to anecdote can also sometime be camouflaged with euphemisms. We see this treatment working individually, even if it cannot be demonstrated with group level data. Translation – this treatment does not work when studied scientifically, so I am going to refer to anecdotes instead.
This approach can be further obscured by referring to individualized or personalized medicine. These sound good, because they are, but the problem is – you should not be individualizing treatments that have no proven efficacy. When I give a patient a treatment trial, I give them something that has been proven statistically to work, and then I see if it works in them (you still never really know, you can only know the statistics). What you cannot do is use individual level data as if it were evidence for efficacy.
Oh, yeah!
One final strategy is to say that, well, mainstream medicine is not all evidence-based either. This is actually a complex question, and not statement like this can fairly capture that complexity. In scientific medicine we use a hierarchy of evidence, starting with the most evidence based treatments (individualized by history, comorbidities, and priorities), and then proceed from there as necessary. At the same time we are trying to improve the evidence base for everything we do. The vast majority of day-to-day medical practice is solidly evidence-based, but we don’t have proven treatments for everything, so we often have to make do with less-than definitive evidence.
However, there is a line we should not go below. We should not use treatments that have been adequately demonstrated to be harmful or worthless. We should also not use treatments that have no evidence to support their use and have a prior probability of close to zero. We also should not use treatments that have been studied for decades without being able to prove specific efficacy, and just pretend these are still experimental.
This also gets us back to my first point – the goal of scientific medicine it to improve the base of scientific evidence for medicine, to improve the standard of care, and to increase the effectiveness of translating evidence into best practice. Yes, medicine is not perfect. But like all sciences, it is self-correcting. While there is a major movement within medicine to constantly tweak and improve the scientific base, there is a simultaneous movement that seeks to carve out a double standard that explicitly lowers the standard of evidence, of science, and even of ethics within medicine.
At the end of the day, the CAM movement is all about integrating old, unscientific, failed treatments into modern medicine. Everything else is distraction.

Is There a Replication Crisis?
The true causes and implications of the replication problem in science.
Steven Novella on June 8, 2022
Link

The replication “crisis” in medicine, and science more generally, has been the poster child for problems with how science is conducted since 2015. That is when the term first appeared, following a published attempt at replicating 100 psychological studies. The authors found that only 39% could be replicated, meaning that the other 61% were probably spurious, and the original studies false positives. As we have been pointing out since then, calling it a “crisis” is a bit dramatic, but it is a legitimate issue. Properly framed, it can be a good lens into the challenges of modern science.
A replication is a study that attempts to reproduce the findings of an earlier study. Sometimes this can be an exact replication, where the methods are followed precisely on a fresh set of data to make sure the original findings were not a statistical fluke. Other replications try to reproduce the same phenomenon, but with slightly or even completely different methods. The idea is that if the phenomenon is real it should replicate for anyone who looks for it. We can also get more technical and say that the research should find a statistical Bell curve of probability around the true effect size.
Wrong ideas, or hypotheses that are false, should also display a statistical curve of probability, which means that there are going to be false positives. So really when we talk about the replication crisis (let’s call it a “problem” from here out) the real question is – if science is functioning optimally (no fraud, no mistakes, valid methods), what should be the rate at which initial positive findings replicate with later research? It’s not 100% because not all new hypotheses in science are correct and even false hypothesis should generate positive data sometimes by random chance. Is 39%, as was found by that original psychological paper, a reasonable number? If so, then there is no problem at all and we are wringing our hands over nothing. 
A recent article by Alexander Bird argues that this latter position may be close to the truth. He writes:
In this article I argue that the high rate of failed replications is consistent with high-quality science. We would expect this outcome if the field of science in question produces a high proportion of false hypotheses prior to testing.
The apparent replication problem, he concludes, is therefore just a manifestation of the base rate fallacy. This is a failure to consider the base rate at which a phenomenon occurs when thinking about how likely a specific instance should be. In some contexts this may also be called the representativeness heuristic (a cognitive bias based on the base rate fallacy). For example, anti-vaxxers may point to the fact that there are more people who are vaccinated who get COVID than people who are not vaccinated, implying that the vaccine does not work. However, this fails to consider that there are many more people who are vaccinated than unvaccinated – the base rate.
In the context of replication, the base rate fallacy results in underestimating the number of new hypotheses that are false. That really is the ultimate question here – of all new ideas in science, how many of them will turn out to be true. If that proportion is small, then most studies will be looking at an ultimately false hypothesis. Therefore the false positive studies from the wrong hypotheses may turn out to be higher than the true positive studies from the smaller proportion of correct hypotheses.
So what’s the answer – how many new hypotheses in science are actually correct? We don’t know specifically, and this likely varies widely across disciplines, cultures, and even institutions. There have been estimates, however, with answers of around 10%. That is a representative figure, but we can use it to do some thought experiments. In a world where 10% of new ideas in science are correct, and where the conduction of science is perfect, and we use a 0.05 P-value as the cut off for a “positive” study, and further all studies are statistically powerful enough to detect the phenomenon, then about a third of positive studies will be false positives and won’t replicate.
But of course we don’t live in a perfect world. The one third figure is actually lower than some studies are finding, in terms of original studies that don’t replicate. This is why some researchers argue that, even if we account for the base rate fallacy, that does not completely explain the replication problem. It does, however, put it into context and support the position that the replication problem may be overblown. Still, we should be moving toward a more optimal scientific world. How do we get there?
We have addressed this issue many times at SBM, so here is a quick summary (all proposals by other scientists and published in the literature). Some authors argue that we should lower the traditional P-value threshold for accepting a finding as “statistically significant”, from 0.05 to 0.005. This would reduce the false positive rate but increase the false negative rate – this is unavoidable, the question is, where is the optimal balance? Perhaps 0.05 is not it. The deeper question here is, are false positives more a problem for science or false negatives? Should we have a high tolerance of false positives (in which case, stop complaining about the non-existent replication problem), or are they a drag and inefficiency in the system that we should seek to minimize? On the other hand, if a false negative kills a good idea in the crib, is that a greater loss for science?
Others argue that we should stop relying so much on P-values. Journals should (and some are) requiring other measures of if a hypothesis is likely true or not, such as effect sizes, and Bayesian analysis. If we use multiple measures of probability, then perhaps we can lower both the false positive and false negative rates.
Yet another proposal is for internal replications to become standard in scientific studies. This means that researchers will test a new hypothesis with a preliminary set of data, and if positive then replicate the findings themselves with a fresh set of data. If this internal replication works, then publish. This way we can reduce some of the false positive noise in the literature without increasing the false negative rate.
This all sounds fantastic, but critics have pointed out that as a pragmatic issue there are problems here. Raising the bar significantly for getting published could be crippling for young researchers trying to get their PhD or launch their career. This would be especially onerous on marginalized groups and researchers from poorer countries or institutions.
So how do we balance all of this in our theoretically perfect scientific world? One way is to institutionalize (to an even greater degree than already is the case) scientific best practices. This should include all the things we discuss here, including preregistration of trials, eliminating behavior that results in P-hacking, and including more thorough statistical analysis rather than relying solely on P-values. Editors of scientific journals also need to stop favoring articles just for their impact factor, and publish more replications. Institutions need to support young researchers, and adjust their “publish or perish” culture that motivates publishing lots of preliminary data.
Even with these tweaks, there are going to be many published studies that are false positive and don’t replicate. It comes with the territory, as Bird pointed out, and is statistically unavoidable. Doing internal replications is great, but is unrealistic to have the lowest bar for publication being so high. As a compromise I propose that journals publish preliminary research, as they do now, but label it as such. This is a warning to journalists and other researchers – these are preliminary data about a fresh hypothesis, many or most won’t replicate, so view these results with caution. But this also gets the new hypothesis out there for others to research. Remember – whether or not a finding replicates is how we find out if a phenomenon is real. If 100% of findings replicated, then it would not be serving that purpose.
The real replication problem, therefore, may be mostly one of perception. The problem with findings that don’t replicate is not that the original study was a false positive, or the new hypothesis was wrong. Again, these are unavoidable and part of the scientific process. The problem is a media culture that presents every new finding as if it is definitely real and we should immediately change our behavior based on the new finding. This is exacerbated by social media, and by an industry of hype and snake oil looking to exploit such findings. A prominent warning about the preliminary nature of such findings may help mitigate this.

The Null hypothesis: Gary Null attacks science-based medicine
Over the last couple of weeks, one of the old men of quackery, Gary Null, has decided (yet again) that he really, really doesn’t like science-based medicine. That includes Steve Novella, Susan Gerbic, and…me. As is his usual habit, Null teamed up with his producer Richard Gale and wrote some seriously off-base screeds against Wikipedia, skeptics, and science-based medicine, basically the forces that keep his beloved homeopathy down.
David Gorski on May 14, 2018
Link

Sometimes, a topic appeals to me because I think it would be fun to write about, even if there are other stories in the news or studies recently published that might be more serious or perhaps more interesting to more of our readers. Sometimes a topic demands to be discussed simply because I feel as though it is my responsibility or duty to do so. Sometimes, on uncommon occasions, a topic falls into both categories. So it is this week, because Gary Null has been attacking science-based medicine in general and even naming names, including Steve Novella and myself, among others. Since Steve appears to have declined to pick up the gauntlet (and might even be wise to have done so), I have a hard time letting something like this pass without a response, because, I guess, that’s just who I am.
Who is Gary Null?
Oddly enough, we haven’t really mentioned Gary Null much on SBM over the years, at least not compared to other media promoters of quackery and pseudoscience. I say “oddly enough” because Null has been doing what he does a very long time. You might recall (or not) that five years ago I featured Gary Null because back in the late 1970s he was publishing articles promoting alternative cancer cures in, of all places, Penthouse. Since then, he’s been pretty much continuously doing the same thing, publishing articles and books promoting alternative medicine and bashing conventional medicine, and, more recently, running a radio network, the Progressive Radio Network (PRN.FM), where his bio describes him thusly:
An internationally renowned expert in the field of health and nutrition, Gary Null, Ph.D is the author of over 70 best-selling books on healthy living and the director of over 100 critically acclaimed full-feature documentary films on natural health, self-empowerment and the environment. He is the host of The Progressive Commentary Hour and The Gary Null Show, the country’s longest running nationally syndicated health radio talk show which can be heard daily on here on the Progressive Radio Network.
Throughout his career, Gary Null has made hundreds of radio and television broadcasts throughout the country as an environmentalist, consumer advocate, investigative reporter and nutrition educator. More than 28 different Gary Null television specials have appeared on PBS stations throughout the nation, inspiring and motivating millions of viewers. He originated and completed more than one hundred major investigations on health issues resulting in the use of material by 20/20 and 60 Minutes. Dr. Null started this network to provide his followers with a media outlet for health and advocacy. For more of Dr. Null’s Work visit the Gary Null’s Work Section or Blog.GaryNull.com In addition to the Progressive Radio Network, Dr. Null has a full line of all-natural home and healthcare products that can be purchased at his Online Store.
Because of course they can and of course Null sells lots of supplements. Amusingly (and so ironically) back in 2010, Null actually poisoned himself with his own supplements. Actually, it’s only funny in retrospect because we know Gary Null recovered and has been fine, but it’s hard not to note that own supplements almost killed him through a vitamin D overdose. In any event, for over 40 years, Gary Null has been a force in alternative medicine, antivaccine pseudoscience, and, since the AIDS epidemic, HIV/AIDS denialism (although of late he’s been backpedaling a bit seven years after his denialist propaganda movie AIDS, Inc.). In recent years, we’ve taken note of him for his articles, books, and “documentaries” with titles like Death by Medicine (movie on YouTube), Seeds of Death: Unveiling The Lies of GMO’s (because of course Null is anti-GMO), The Law of Attraction (because of course he believes in The Secret), Preventing and Reversing Cancer Naturally, Autism: Made in the USA (because of course he believes vaccines cause autism), and many more.
As is the case for many quacks and advocates of quackery like coffee enemas, Null’s PhD appears to be of questionable provenance. He received a PhD in Interdisciplinary Studies from Union Institute & University, which is a correspondence college accredited by the North Central Association of Colleges and schools, but only for the humanities. Steve Barrett provided a detailed look at Null’s education on Quackwatch, which led to Null threatening a libel suit. (Yes, like many supporters of quackery, Null is notably litigious.) Sometimes, though, he just likes to counterattack using his media sources, such as the time that Gary Null and Sayer Ji spent much of Ji’s appearance on Null’s radio show attacking science-based medicine and myself.
However, I’ve never seen Null take three or four episodes of his radio show to attack his enemies, which is what he did recently.
Gary Null attacks
One of the first things one learns perusing the PRN.FM website for its recent content is that he really, really doesn’t like Wikipedia. In fact, he features a really long letter to the Wikimedia Foundation from his lawyer Neal S. Greenfield demanding that Wikipedia alter its entry on Null because, according to Null, it uses unreliable “skeptic” sources and characterizes him as a crank. I can’t help but post a couple of excerpts from the article that are quite hilarious to anyone familiar with Null:
The TIME article, “The New Mister Natural” does not include any language indicating that Gary Null denied that HIV causes AIDS. The article does state that Gary believes that, “traditional therapies are ineffective [because] it has never been proved that HIV plays as great a role in the disease as scientists believe.”
The clause, “denies that HIV causes AIDS” is a lie, as the cited TIME magazine does not support the claim.
So Null basically said that he didn’t think HIV plays a central role in the pathogenesis of AIDS, but that calling him an HIV/AIDS denialist is a lie. I’m sorry, Mr. Null, but the very definition of an HIV/AIDS denialist is someone who downplays beyond what any science will support or even outright denies that HIV causes AIDS, which is what you did. Even in the denial he cites, he waffles:
Because I challenged the safety and efficacy of AZT and called into question the science behind conventional AIDS treatment, I was personally attacked as an AIDS denialist, which I categorically deny. There is no debate that HIV exists and that it attacks the immune system. What is still up for debate is whether pharmaceutical drugs are the complete answer. Back then and still today, there is no opportunity for questioning, you are either on board or thrown overboard and then attacked.
Notice how Null only admitted that HIV “exists” and “attacks the immune system.” He doesn’t say categorically that HIV causes AIDS. Then he pivots into a rant about not being able to “question” the HIV/AIDS paradigm. In the letter, this continues:
Here, as in the other parts of the BLP, the authors/editors show malice aforethought and intent to defame. They do so behind a curtain of anonymity, relying on false names and the perceived impossibility of being sued for their libelous statements. These statements were made and reinforced over a period of time by similarly minded people, conspiring with each other, as a Cabal. The question then arises, “why?” Certainly not to report facts or employ truth or fairness.
Yes, to Null it’s all a big conspiracy among skeptics and supporters of science, not individuals encountering his pseudoscience and being appropriately appalled. Leading the conspiracy, according to Null, appear to be two people with whom SBM readers should be acquainted: Stephen Barrett, MD (whom Null really, really, really hates) and Susan Gerbic, with a lesser role played by our fearless leader Steve Novella and yours truly.
Null vs. Steve Barrett
There is an article by two authors listed only as “S.L. and R.A.” entitled, Stephen Barrett and Medical McCarthyism. In it, Null’s flunkies show that Dr. Barrett and his Quackwatch website must be very effective indeed, because Null has been attacking Barrett at least since 2005. The authors basically admit as much in the introduction, while hilariously discussing how they will counterattack using…reason and science:
He has authored reports on many of the most accomplished practitioners and experts in the alternative health movement and in doing so, has generated a fair amount of controversy and mixed media attention. So much, in fact, that the reports on his site have come to dominate web search engine results, and in effect, preemptively tainting the reputations of hundreds of legitimate, well credentialed alternative health practitioners. Upon discovering this, my curiosity was piqued and I felt compelled to conduct some independent research on the matter, and hopefully, reach a conclusion as to whether Barrett was, indeed, an expert, or guilty of what Dr. Eng describes as “medical McCarthyism”.
My focus would be the history and relationship between Barrett and Dr. Gary Null. Null is arguably the most respected, prolific advocate and high-profile voice in the alternative heath movement, influencing a massively wide spectrum of people throughout a varied host of philanthropic efforts and causes. The purpose of this paper is to not to bring direct challenge to Barrett’s work or ideology, but rather to present facts and convey reasoned, journalistic interrogation into the heart of this debate. To that end, we can look to Null’s extensive work and research on the negative effects of fluoride, mercury, vaccines, sugar and caffeine, all of which, Barrett has called in to question. Research will demonstrate that science firmly supports all of Null’s conclusions and solutions on these topics. Fact checking and research is the cornerstone of the journalistic process, yet, Barrett and those media outlets who would employ his subjective opinion as scientific fact, quite simply, have not done their homework here.
Yes, indeed. Null’s toadies appear to be basically admitting that Null is unhappy with Barrett because his articles about Null rank high on Google searches. They also don’t like how media outlets not infrequently seek out Dr. Barrett for interviews when researching stories on alternative medicine:
CNN, The New York Times, and other traditional, highly esteemed news outlets frequently cite Stephen Barrett as an expert in the discussion of the effectiveness and validity of Alternative Health, be it acupuncture, homeopathy, nutritional support, or chiropractic. Barrett’s primary strategy in his campaign is to attack, and in certain cases, bring suit against, key members and pillars of the alternative health movement. One such target is Dr. Gary Null. Barrett’s claims that Null “promotes hundreds of ideas that are inaccurate, unscientific, and/or unproven….” are plainly false. Amongst those ideas are that the intake of Fluoride is harmful and potentially deadly, and that mercury in dental fillings can have serious neurotoxic effects. Dr. Null has also consistently warned of the harmful impact of sugar and the negative effects of caffeine. All of his observations and conclusions are supported by extensive, peer reviewed research and hard-won scientific scholarship. This approach stands in stark contrast to Barrett’s own fast and loose, “things don’t need to be tested [because] they simply don’t make any sense” methodology.
I couldn’t help but laugh at the characterization here. A better way to put it would be, just as quacks, antivaxers, and pseudoscientists do, Null cherry picks the literature to find dubious studies that support his preconceived notions.
As for that last part about “things don’t need to be tested [because] they simply don’t make any sense,” that’s clearly a straw man version of the main precept of science-based medicine. Basically, Null mischaracterizes the SBM view about, for example, homeopathy as being that it doesn’t warrant further scientific testing because it “doesn’t make sense.” While that is the argument—although it would have been more accurate to say, “doesn’t make sense scientifically“—it is an intentionally incomplete statement of the SBM argument that makes it sound as though skeptics dismiss pseudoscience like homeopathy out of hand just because it “doesn’t make sense.” In actuality, the correct argument is that homeopathy’s precepts, particularly the law of infinitesimals, which states that diluting a homeopathic remedy makes it stronger, violate multiple longstanding well-supported laws and theories of science. As I like to say, for homeopathy to “work,” not only would much of what we know about physics, chemistry, and biology have to be wrong, it would have to be spectacularly wrong. As a consequence, the prior probability, from a Bayesian perspective, that homeopathy can work, is so small as to be, well, homeopathic. The burden of evidence thus falls on homeopaths and believers in homeopathy (like, as you will soon see, Gary Null) to provide evidence from the basic sciences that show that homeopathy could actually work, something they as yet have failed to do.
I also couldn’t help but laugh out loud when I read the conclusion:
I can conclude that with absolute proof of an outstanding educational background, and his extensive clinical experience, that Gary Null is being attacked for what he represents: a viable challenge to the existing medical paradigm.
You know, that last line makes me wonder whether Gary Null actually wrote this article. It doesn’t matter. He wouldn’t have published it if he didn’t support its conclusions.
Gary Null vs. Susan Gerbic
More recently, Null has been very unhappy about Wikipedia, a taste of which I provided above from his lawyer’s letter to the Wikimedia Foundation. The result has been a two part screed, Wikipedia: Our New Technological McCarthyism, Part 1 and Part 2. It’s primarily in Part 2 where he lashes out at Gerbic.
It’s not at all surprising that Null would not appreciate Gerbic’s efforts, given that she’s created an organization, Guerilla Skepticism on Wikipedia (GSoW) dedicated to making sure that Wikipedia articles on topics with which skeptics concern themselves are scientifically accurate and that promoters of pseudoscience and quackery don’t manage to bias Wikipedia articles in favor of nonsense. Her group filled a definite need, and she answered the call. Indeed, the slogan of GSoW is, “The mission of the Guerrilla Skepticism editing team is to improve skeptical content of Wikipedia. We do this by improving pages of our skeptic spokespeople, providing noteworthy citations, and removing the unsourced claims from paranormal and pseudoscientific pages. Why? Because evidence is cool. We train – We mentor – Join us.”
Of course, this is exactly the sort of thing that quacks cannot abide, which is why they’ve attacked Gerbic before. For instance, five years ago, Deepak Chopra was very upset at the GSoW treatment of woo-meister Rupert Sheldrake. As I pointed out at the time, there is indeed a movement afoot to correct scientific misinformation and try to keep the entries in Wikipedia scientifically accurate. Chopra complained about this as though it were a bad thing, which to him it was, because Susan’s activism keeps entries that have anything to do with the sorts of quackery and pseudoscience he believes in from being transformed by quacks to glorify and promote those selfsame forms of quackery and pseudoscience—or at least from presenting them as though they were true or even just as scientifically valid points of view.
So what does Null think of Gerbic and her activities? He’s not a fan:
One group that has received Wikipedia’s full support and swallowed Dawkin’s “militant atheism” whole with steroids is Guerrilla Skepticism on Wikipedia (GSoW), founded by Susan Gerbic. GSoW actively seeks out and trains recruits to serve as an army of a Skeptic editors to wage wiki wars against those who research or advocate alternative medical treatment modalities. These are the most active of Wikipedia’s independent advocates editing alternative medicine content and pages critical of conventional drugs.
I can’t help but point out here that there was a previous version of this article in which this paragraph read a bit differently:
One group that has received Wikipedia’s full support and swallowed Dawkin’s “militant atheism” whole with steroids is Guerrilla Skepticism on Wikipedia (GSoW). Founded by a woman who owns a small portrait gallery in Monterrey, Susan Gerbic is a close friend of SBM guru David Gorski who fully endorses her organization’s internet militancy. GSoW actively seeks out and trains recruits to serve as an army of a Skeptic editors to wage wiki wars against those who research or advocate alternative medical treatment modalities. These are the most active of Wikipedia’s independent advocates editing alternative medicine content and pages critical of conventional drugs. To further proselytize her extreme Skepticism globally, Gerbic later founded the World Wikipedia Project to reproduce her successes on foreign language Wikipedias.
Why was the change made? I can’t know for sure, but about a week and a half ago my not-so-super-secret other blog noted many errors of fact in Null’s articles. As a result, the articles mysteriously disappeared for a couple of days, and when they reappeared they had been…edited. Also, Null appears to have removed several mentions of my name, perhaps because he was embarrassed, perhaps because he was ticked off. For instance, in the original version of Part 1, Null claimed that I was based at Wake Forest University. Truly, I take what I view to be as deserved credit for “helping” Null improve his accuracy, although I will likely point out more of his errors of fact before this post is done. For instance, one error that didn’t get corrected was that the World Wikipedia Project only existed from 2012-2014 and was merged into GSoW.
Elsewhere:
On her personal Wikipedia biographical entry, Gerbic is quoted as saying, “We rewrite Wikipedia, and proof the pages, we remove citations that are not noteworthy, we add citations, we do just about everything in Wikipedia to improve content.”[13] Of course, the majority of their “notable citations” reference back to Skeptic and SBM sources, such as Gorski’s ScienceBasedMedicine blog. “Improvements” are solely aligned with Skepticism’s doctrine. Gerbic’s other organization Skeptic Action is another stealth guerilla operation to disseminate cyber tasks for Skeptic trolls on Twitter, Facebook and Google+ to rapidly rate pages such as books listed on Amazon that question vaccination, homeopathy, and natural cancer treatments. Skeptic Action also utilizes a community drive system, which enables members to receive rapid alerts to rebut content posted on the internet.
Gee, Null says that as though it were a bad thing. That’s not surprising, because Wikipedia would be a great marketing tool for quacks, and a team of skeptics preventing pseudoscience and dubious references from being inserted into Wikipedia entries is a major buzzkill when it comes to shiny happy propaganda about alternative medicine. Indeed, Null even inadvertently provides an example:
In this particular case, Debby Vajda, President for the Association for Comprehensive Energy Psychology (ACEP), provided 51 peer-reviewed articles and studies, 18 which were randomized controlled studies, appearing in professional journals, including the American Psychological Association, the Journal of Clinical Psychology, the Journal of Nervous and Mental Diseases, Psychotherapy Theory Research and Practice and others showing positive statistical results outside the range of chance. She commented on Change.org, “Every edit to the energy psychology Wikipedia page that attempts to reference findings from these well-respected, scientific journals is summarily deleted… The American Psychological Association does not think we are ‘lunatic charlatans.’ Neither does the Association of Social Work Boards, the National Board of Certified Counselors, or the National Association of Alcohol and Drug Abuse Counselors, all of which approve ACEP to provide continuing education to their professional members for the study of energy psychology. The Wikipedia page is out of step with existing peer-reviewed research on this topic, and opinionated, self-described “skeptic” editors are resisting any change.”[18]
Apparently the scientific evidence was insufficient to pass Wikipedia’s administration review. The page still defines Energy Medicine as a “pseudo-scientific belief.”[19]
See? Energy medicine “quacks” are trying to promote their quackery with the usual dubious pseudoscientific studies. Fortunately, Wikipedia is quite correct to define energy medicine as pseudoscience. As I like to say about reiki, for instance, it’s faith healing substituting Eastern mystical religious beliefs for the Christian beliefs usually used to justify faith healing. A list of the sorts of articles ACEP promotes as “rigorous” include the usual nonsense about Emotional Freedom Technique, acupuncture, thought field therapy, “Wholistic Hybrid Derived from Eye Movement Desensitization and Reprocessing and Emotional Freedom Technique,” and the like. (EFT, which is pure wishful thinking quackery, seems to be very big among this group.)
Null also blames it all on “militant atheism”:
The Center for Inquiry (CFI), the umbrella organization that serves as the mother chapel for the Skeptic movement, fully embraces Dawkins’ atheistic Scientism. In 2016, the Richard Dawkins Foundation for Reason and Science merged with CFI. Its stated mission is to “foster a secular society based upon reason, science, freedom of inquiry and humanist values.”[12] Laudable words, but the Center fails horribly to tolerate, let alone respect, the freedom of others to their beliefs and the freedom to choose a medical intervention of their choice. Any discipline of inquiry that is performed outsides the Center’s narrow interpretation of science is condemned as heresy, exposed and publicly maligned. Everything that deals with religion and spirituality, the paranormal, unexplained phenomena, and alternative and natural medical modalities are accused of con-artistry. Other leading major Skeptic groups are the Committee for Skeptical Inquiry, the Council of Secular Humanism, the James Randi Educational Foundation and the SBM-related Commission for Scientific Medicine and Mental Health.
The latter publishes the Scientific Review of Alternative Medicine, founded by Skeptics at Stanford University and the Committee for Scientific Investigation of Claims of the Paranormal. The publication makes the narcissistic claim of being the only journal that properly analyses alternative medical claims.
I do love how out of date Null’s information is. For instance, the James Randi Educational Foundation has basically gone into hibernation. Sure, it still exists, and supposedly has converted itself into a grant-making foundation, but I haven’t heard much of anything about such grants being awarded since the JREF made that change. Indeed, its website appears not to have changed much two and a half years, the sole recent announcement since 2015 being in January, in which JREF announced that Susan Gerbic received the 2017 award from the JREF, saying that the award “is given to the person or organization that best represents the spirit of the foundation by encouraging critical questions and seeking unbiased, fact-based answers.” Susan confirmed this award to me, and it cheered me up a bit to learn that JREF isn’t completely moribund.
As for the rest, does Null not know that the Commission for Scientific Medicine and Mental Health unfortunately has been moribund for years. The most recent articles on its website are from 2005, for cryin’ out loud! The most recent issue of Scientific Review of Mental Health Practice (SRMHP) was published in 2007. The most recent issue of the Scientific Review of Alternative Medicine (SRAM) was published in 2002. It’s sad, yes, very sad, and I do know that every now and then someone brings up the idea of trying to resurrect SRAM and SRMHP, but in reality they are probably no longer necessary given the rise of blogs and online publications. That doesn’t stop Null from attacking them as though they were still relevant.
Gary Null versus SBM on behalf of homeopathy
If there’s one thing that seems to tick off Gary Null, it seems to be the opposition to homeopathy by supporters of science. He mentions it on multiple occasions in his “McCarthyism” articles and most recently published an article co-authored with Richard Gale (the Executive Producer of the PRN.FM) entitled Slaying Homeopathy and Americans’ Freedom of Health Choices. It begins, as many of Null’s articles begin, with an attack on the medical profession as being against—of course!—freedom:
A fundamental foundation of American democracy is our freedom of choice. We have the legitimate right to pursue a career, decide where to live, what to read or watch, the freedom to vote or not, and decide who to marry or live with. Americans hold this these freedoms sacred, at least in theory.
However, during the past decades, an ethos has arisen, associated with the American Medical Association, the pharmaceutical industry, professional organizations, medical publications, media health reporters and legislators largely benefitting from the largess of private drug companies adamantly pushing to limit our medical choices. This medical regime wants to choose and dictate the types of medicines available and enforce how we are diagnosed for mental conditions. They have mastered the art of pathologizing life. We have little choice in having access to public water without fluoridation or the right to vaccinate our children or not. To illustrate this point, the Department of Health and Human Services (HSS) and FDA curtailing all positive information about homeopathy and making efforts to eliminate it altogether. This is accomplished under the misinformed ruse that it is unproven and has not undergone the rigor of conventional medical science. Therefore, homeopathy should be banned.
Yes, it is true that the FDA and FTC have of late made promising moves towards regulating homeopathy and its advertising just like other over-the-counter drugs. Yes, to those of us supporting SBM, this is a good thing, although I personally think that the FDA and FTC haven’t gone far enough.
Naturally, Null doesn’t like skeptics:
If we listen solely to the Skeptics – a quasi-rationalist movement of scientific idolatry — and the advocates of Science-Based Medicine (SBM), and now increasingly government health officials, homeopathy’s explanation is always the same. Homeopathy is only water. It is foolish and unscientific. Worse, we are told it is dangerous and even life-threatening. Homeopathy, so the Skeptics believe, fails to meet the standards of modern evidence-based medicine and should therefore be avoided. That is the end of the story.
Would that it were true that government health officials are increasingly viewing homeopathy as The One Quackery To Rule Them All, which it is! Sadly, despite promising moves here in the US and across the pond in the UK, in a lot of other countries homeopathy is almost as accepted as real medicine. For instance, recently in France a group of physicians has been harassed with complaints to French medical authorities for daring to speak out against the government’s support of homeopathy. Meanwhile, Agnès Buzyn, ministre des Solidarités et de la Santé (Minister of Solidarity and Health), stated her support for homeopathy, specifically, “If it continues to be beneficial, without being harmful, it will continue to be reimbursed.” This is a pretty damned irresponsible thing for any minister of health in any country to say. Don’t believe me? Check this out:
“Si ça continue à être bénéfique, sans être nocif, ça continuera à être remboursé”➡ @agnesbuzyn ministre des Solidarités et de la Santé#BourdinDirect #Santé #homeopathie pic.twitter.com/OsfsmbG5yP
— RMC (@RMCinfo) April 12, 2018
In Germany, homeopathy remains popular as well. Ditto India and a lot of other countries. Even though homeopathy is no longer supported by the NHS the way it was, the royal family still uses it, and Prince Charles is its champion. Indeed, Null uses the popularity of homeopathy in other countries to claim that it must have some validity.
Predictably next up, likening SBM to a religion:
The priests in this Citadel who serve the scientific and technological elite of postmodernity, namely federal health agencies and the pharmaceutical industrial complex, in fact properly speak their truths about homeopathy and other non-conventional medicine. But their truths only exist within a very specified, narrow, and limited context or field from which they criticize science they either don’t or refuse to understand or are incapable of accurately explaining. For this reason they are easy to criticize because in fact they are the self-anointed spokespersons for the dire failings of modern scientific culture, conventional medicine and repressive government health policies.
Dr. Steven Novella, the acknowledged founder of Science-Based Medicine describes homeopathy as;
“a prescientific philosophy-based system on magical thinking…. Essentially, homeopathy uses fanciful treatments that are based on silly ideas, such as the personality of the patient, but also “sympathetic magic.” The belief is that homeopathic remedies contain the magical essence of symptoms and can be used to cure those same symptoms.”[3]
Another spokesperson for the SBM movement is oncologist and breast cancer surgeon Dr. David Gorski (aka ORAC). Gorski writes, “Homeopathy is the perfect quackery. The reason that homeopathy is so perfect as a form of quackery is because it is quite literally nothing.”[4] But are the Skeptics perhaps accurate in their assessment about homeopathy? Is over-diluting a substance in water whereby the atoms are no longer detectable any more bioactive than plain water? Yes, we need to ask these questions. But it is also wise to parallel this inquiry and ask whether most conventional drug therapies safely work and properly improve and save lives.
Yes, these are all familiar tropes. Medicine is a religion, and, oh, by the way, conventional medicine doesn’t work. After this follows a litany of all the clinical trials of homeopathy that have “proven” that homeopathy is safe and effective. Naturally, none of them are convincing if you really look at them with a skeptical eye. How often have we here dissected various clinical trials of homeopathy and shown that they do not show what adherents think they show? Null also ignores that one has to examine the totality of the scientific evidence, basic, preclinical, and clinical. After all, by their very nature, clinical trials occasionally produce false positive results, which means that homeopaths and their supporters (like Null) can always manage to cherry pick some studies with weak statistically significant results.
Elsewhere, in part 2, Null actually claims that skeptics do not understand homeopathy. Indeed, he views homeopathy as an example of what’s wrong with skepticism:
Homeopathy is an excellent example of Skepticism’s unsound and frequently unsubstantiated criticisms. Simply because SBM physicians may not understand biophysics, quantum energy, and physical properties of water should not close the door on homeopathy as mere quackery. Surely Skeptics will embrace the value of nanotechnology without understanding the physics of spatial quantum confinement behind it. Nanomedince is rapidly becoming part of conventional medicine’s drug arsenals. Safety studies for nano-drugs are weak at best. Yet there are analogous features to nanotechnology and homeopathic theory in terms of spatial physics and force. Furthermore, in Europe, homeopathy is a preferred alternative treatment modality among doctors. In India, where it is most popular, 62% of homeopathic users have never tried conventional drugs, and 82% of those in an AC Nielsen survey said they would not switch to allopathic treatments. In France, 94% of surveyed pharmacists acknowledged they recommend pregnant women to use homeopathic remedies instead of pharmaceutical drugs. Homeopathy is also taught in 21 of 24 French pharmacology schools. Seventy percent of French physicians approve of the discipline.[25]
And, in Slaying Homeopathy:
We have identified several possible reasons. First, American physicians, and notably the Skeptics, who oppose homeopathy and other scientifically proven alternative medical modalities, are not intellectually capable of understanding the principles of homeopathy and hormesis. Skeptics are victims of their own “strange loops” — a term coined by Douglas Hofstadter, the Pulitzer Prize winning professor of cognitive science and mathematics.
“Strange loops” can account for the paradoxical “irrational rationality” so prevalent in Skeptics’ faulty reason when ranting in their comic diatribes against homeopathy, acupuncture, CAM, etc. Skeptics are victims of scientific ignorance and exist within a local tribal tunnel-reality of their own creation.
Second, and alternatively, SBM advocates are fully aware of the scientific literature supporting homeopathy but have been institutionally conditioned to be biased against it. And third, they are fully aware but homeopathy competes with their own preferred conventional therapies. The first possibility is undoubtedly true for Skeptic organizations, such as the Committee for Skeptical Inquiry, and its advocates. The second and third rationales are likely the case for the medical professionals who follow Science-Based Medicine, and Skeptics like Craig Pearcey who have a very sound grasp of many of homeopathy’s underlying principles as well as quantum mechanics. Allowing SBM’s spokespersons the benefit of the doubt that their blogs and guerrilla tactics are not directly receiving funds from Big Pharma and other private or professional medical interests, nevertheless they all summarily choose to deny any scientific evidence to support homeopathy’s value.
Shout out to Craig! I also “thank” Null for admitting the possibility that we SBM skeptics aren’t all pharma shills, although one can’t help but note that he certainly implies it while “giving us the benefit of the doubt.”
Actually, it is because SBM-supporting physicians do have an understanding of biophysics, quantum energy, and physical properties of water that we reject homeopathy as pseudoscience. It is because quacks like Null do not understand these things and are unduly impressed by the shiny, happy pseudoscience and mysticism invoking quantum physics as though it were magic. Also, one more time, homeopathy has nothing to do with nanoparticles, and appealing to homeopathy’s popularity is meaningless. You can be sure that Null won’t mention that homeopathy is big business in France (Boiron, anybody) and that powerful forces actually suppress criticism of homeopathy by SBM physicians. Yes, in France, criticizing homeopathy too vigorously can land you in a world of hurt, including complaints against you and even lawsuit, and the Minister of Solidarity and Health fully supports homeopathy. In France, homeopathy is mainstream. To SBM, that’s a problem. To Null, it’s a good thing.
The Null hypothesis
Although his star has been eclipsed by younger interlopers such as Mike Adams, having been at his chosen profession for well over 40 years now, Gary Null nonetheless remains influential among proponents of quackery, even though his methods are very old school (articles, radio shows and podcasts, and movies) and his voice, whenever I dare try to listen to his radio show, makes an excellent nonpharmacological insomnia aid, which no doubt Null would approve of over something like Ambien. For as long as I’ve paid attention to him (about a decade now), his M.O. has been basically the same and consists mainly of:
    • Attacking conventional medicine as dangerous and unscientific.
    • Cherry picking bad studies of alternative medicine to
    • Crying persecution by government and the medical profession due to his preferred quackery representing a “viable” alternative to medicine.
    • Conspiracy mongering.
    • Ad hominem attacks against his critics as being in the thrall of big pharma or ideologically biased against alternative medicine.
    • Portraying conventional medicine as a religion.
Which brings us to Null’s conclusions.
From part 2 of his McCarthyism article:
Wikipedia is embedded skeptic groups that attack those who would tell us the truth, the guardians of the social media galaxy. We are brainwashed 24-7 without warning. No trepidation. No open debate. We are solely passive consumers in the wiki matrix. Objectivists, as The Economist article notes, functions best when social conditions reinforce a bee-hive mentality. This is what enables Skeptic leaders to cling to their perceptions of intellectual superiority. In the meantime we have a compliant nation, a population obedient and only buying.
Project much, Mr. Null? The people who would most benefit from a compliant nation, a population obedient and only buying, would be Gary Null, Mike Adams, and his fellow quacks. Certainly they’d benefit far more than any SBM proponent would. We hawve our pesky dedication to science that keeps us from taking full advantage of the free market. That’s why financially we’re always at a disadvantage—and not by a little. Null can never admit that, though, because his very business model depends on him portraying himself as an underdog fighting the system for Truth, Justice, and the American Way:
To win the war on medical truth, Skepticism needs to be called out for what it genuinely is, an aberrant form of destructive skepticism masquerading as the arbiters of authentic science. Only by exposing these individuals and organizations, the flaws in their “irrational rationalities,” and the regressive system of healthcare they represent that threatens the well-being of the public, will homeopathy have a chance of gaining the wider acceptance it deserves.
Say the duo who claim that skepticism is a “curious mix of Orwellian fascism and a quirky technological totalitarianism” and the “darker side of Liberalism, with noticeable parallels to Ayn Rand’s Objectivist and autocratic absolutism.” Yes, I do think that they project.
Of course, in the end, the “freedom” that Null and Gale espouse is freedom from the shackles of the pesky science that demonstrates homeopathy (as well as the other varieties of quackery that they both support) to be ineffective. The reason Null and Gale attack Wikipedia and skeptics is not out of some love of The Truth, but because we get in the way of selling their products.

The Null hypothesis: Gary Null attacks science-based medicine
Over the last couple of weeks, one of the old men of quackery, Gary Null, has decided (yet again) that he really, really doesn’t like science-based medicine. That includes Steve Novella, Susan Gerbic, and…me. As is his usual habit, Null teamed up with his producer Richard Gale and wrote some seriously off-base screeds against Wikipedia, skeptics, and science-based medicine, basically the forces that keep his beloved homeopathy down.
David Gorski on May 14, 2018
Link

Sometimes, a topic appeals to me because I think it would be fun to write about, even if there are other stories in the news or studies recently published that might be more serious or perhaps more interesting to more of our readers. Sometimes a topic demands to be discussed simply because I feel as though it is my responsibility or duty to do so. Sometimes, on uncommon occasions, a topic falls into both categories. So it is this week, because Gary Null has been attacking science-based medicine in general and even naming names, including Steve Novella and myself, among others. Since Steve appears to have declined to pick up the gauntlet (and might even be wise to have done so), I have a hard time letting something like this pass without a response, because, I guess, that’s just who I am.
Who is Gary Null?
Oddly enough, we haven’t really mentioned Gary Null much on SBM over the years, at least not compared to other media promoters of quackery and pseudoscience. I say “oddly enough” because Null has been doing what he does a very long time. You might recall (or not) that five years ago I featured Gary Null because back in the late 1970s he was publishing articles promoting alternative cancer cures in, of all places, Penthouse. Since then, he’s been pretty much continuously doing the same thing, publishing articles and books promoting alternative medicine and bashing conventional medicine, and, more recently, running a radio network, the Progressive Radio Network (PRN.FM), where his bio describes him thusly:
An internationally renowned expert in the field of health and nutrition, Gary Null, Ph.D is the author of over 70 best-selling books on healthy living and the director of over 100 critically acclaimed full-feature documentary films on natural health, self-empowerment and the environment. He is the host of The Progressive Commentary Hour and The Gary Null Show, the country’s longest running nationally syndicated health radio talk show which can be heard daily on here on the Progressive Radio Network.
Throughout his career, Gary Null has made hundreds of radio and television broadcasts throughout the country as an environmentalist, consumer advocate, investigative reporter and nutrition educator. More than 28 different Gary Null television specials have appeared on PBS stations throughout the nation, inspiring and motivating millions of viewers. He originated and completed more than one hundred major investigations on health issues resulting in the use of material by 20/20 and 60 Minutes. Dr. Null started this network to provide his followers with a media outlet for health and advocacy. For more of Dr. Null’s Work visit the Gary Null’s Work Section or Blog.GaryNull.com In addition to the Progressive Radio Network, Dr. Null has a full line of all-natural home and healthcare products that can be purchased at his Online Store.
Because of course they can and of course Null sells lots of supplements. Amusingly (and so ironically) back in 2010, Null actually poisoned himself with his own supplements. Actually, it’s only funny in retrospect because we know Gary Null recovered and has been fine, but it’s hard not to note that own supplements almost killed him through a vitamin D overdose. In any event, for over 40 years, Gary Null has been a force in alternative medicine, antivaccine pseudoscience, and, since the AIDS epidemic, HIV/AIDS denialism (although of late he’s been backpedaling a bit seven years after his denialist propaganda movie AIDS, Inc.). In recent years, we’ve taken note of him for his articles, books, and “documentaries” with titles like Death by Medicine (movie on YouTube), Seeds of Death: Unveiling The Lies of GMO’s (because of course Null is anti-GMO), The Law of Attraction (because of course he believes in The Secret), Preventing and Reversing Cancer Naturally, Autism: Made in the USA (because of course he believes vaccines cause autism), and many more.
As is the case for many quacks and advocates of quackery like coffee enemas, Null’s PhD appears to be of questionable provenance. He received a PhD in Interdisciplinary Studies from Union Institute & University, which is a correspondence college accredited by the North Central Association of Colleges and schools, but only for the humanities. Steve Barrett provided a detailed look at Null’s education on Quackwatch, which led to Null threatening a libel suit. (Yes, like many supporters of quackery, Null is notably litigious.) Sometimes, though, he just likes to counterattack using his media sources, such as the time that Gary Null and Sayer Ji spent much of Ji’s appearance on Null’s radio show attacking science-based medicine and myself.
However, I’ve never seen Null take three or four episodes of his radio show to attack his enemies, which is what he did recently.
Gary Null attacks
One of the first things one learns perusing the PRN.FM website for its recent content is that he really, really doesn’t like Wikipedia. In fact, he features a really long letter to the Wikimedia Foundation from his lawyer Neal S. Greenfield demanding that Wikipedia alter its entry on Null because, according to Null, it uses unreliable “skeptic” sources and characterizes him as a crank. I can’t help but post a couple of excerpts from the article that are quite hilarious to anyone familiar with Null:
The TIME article, “The New Mister Natural” does not include any language indicating that Gary Null denied that HIV causes AIDS. The article does state that Gary believes that, “traditional therapies are ineffective [because] it has never been proved that HIV plays as great a role in the disease as scientists believe.”
The clause, “denies that HIV causes AIDS” is a lie, as the cited TIME magazine does not support the claim.
So Null basically said that he didn’t think HIV plays a central role in the pathogenesis of AIDS, but that calling him an HIV/AIDS denialist is a lie. I’m sorry, Mr. Null, but the very definition of an HIV/AIDS denialist is someone who downplays beyond what any science will support or even outright denies that HIV causes AIDS, which is what you did. Even in the denial he cites, he waffles:
Because I challenged the safety and efficacy of AZT and called into question the science behind conventional AIDS treatment, I was personally attacked as an AIDS denialist, which I categorically deny. There is no debate that HIV exists and that it attacks the immune system. What is still up for debate is whether pharmaceutical drugs are the complete answer. Back then and still today, there is no opportunity for questioning, you are either on board or thrown overboard and then attacked.
Notice how Null only admitted that HIV “exists” and “attacks the immune system.” He doesn’t say categorically that HIV causes AIDS. Then he pivots into a rant about not being able to “question” the HIV/AIDS paradigm. In the letter, this continues:
Here, as in the other parts of the BLP, the authors/editors show malice aforethought and intent to defame. They do so behind a curtain of anonymity, relying on false names and the perceived impossibility of being sued for their libelous statements. These statements were made and reinforced over a period of time by similarly minded people, conspiring with each other, as a Cabal. The question then arises, “why?” Certainly not to report facts or employ truth or fairness.
Yes, to Null it’s all a big conspiracy among skeptics and supporters of science, not individuals encountering his pseudoscience and being appropriately appalled. Leading the conspiracy, according to Null, appear to be two people with whom SBM readers should be acquainted: Stephen Barrett, MD (whom Null really, really, really hates) and Susan Gerbic, with a lesser role played by our fearless leader Steve Novella and yours truly.
Null vs. Steve Barrett
There is an article by two authors listed only as “S.L. and R.A.” entitled, Stephen Barrett and Medical McCarthyism. In it, Null’s flunkies show that Dr. Barrett and his Quackwatch website must be very effective indeed, because Null has been attacking Barrett at least since 2005. The authors basically admit as much in the introduction, while hilariously discussing how they will counterattack using…reason and science:
He has authored reports on many of the most accomplished practitioners and experts in the alternative health movement and in doing so, has generated a fair amount of controversy and mixed media attention. So much, in fact, that the reports on his site have come to dominate web search engine results, and in effect, preemptively tainting the reputations of hundreds of legitimate, well credentialed alternative health practitioners. Upon discovering this, my curiosity was piqued and I felt compelled to conduct some independent research on the matter, and hopefully, reach a conclusion as to whether Barrett was, indeed, an expert, or guilty of what Dr. Eng describes as “medical McCarthyism”.
My focus would be the history and relationship between Barrett and Dr. Gary Null. Null is arguably the most respected, prolific advocate and high-profile voice in the alternative heath movement, influencing a massively wide spectrum of people throughout a varied host of philanthropic efforts and causes. The purpose of this paper is to not to bring direct challenge to Barrett’s work or ideology, but rather to present facts and convey reasoned, journalistic interrogation into the heart of this debate. To that end, we can look to Null’s extensive work and research on the negative effects of fluoride, mercury, vaccines, sugar and caffeine, all of which, Barrett has called in to question. Research will demonstrate that science firmly supports all of Null’s conclusions and solutions on these topics. Fact checking and research is the cornerstone of the journalistic process, yet, Barrett and those media outlets who would employ his subjective opinion as scientific fact, quite simply, have not done their homework here.
Yes, indeed. Null’s toadies appear to be basically admitting that Null is unhappy with Barrett because his articles about Null rank high on Google searches. They also don’t like how media outlets not infrequently seek out Dr. Barrett for interviews when researching stories on alternative medicine:
CNN, The New York Times, and other traditional, highly esteemed news outlets frequently cite Stephen Barrett as an expert in the discussion of the effectiveness and validity of Alternative Health, be it acupuncture, homeopathy, nutritional support, or chiropractic. Barrett’s primary strategy in his campaign is to attack, and in certain cases, bring suit against, key members and pillars of the alternative health movement. One such target is Dr. Gary Null. Barrett’s claims that Null “promotes hundreds of ideas that are inaccurate, unscientific, and/or unproven….” are plainly false. Amongst those ideas are that the intake of Fluoride is harmful and potentially deadly, and that mercury in dental fillings can have serious neurotoxic effects. Dr. Null has also consistently warned of the harmful impact of sugar and the negative effects of caffeine. All of his observations and conclusions are supported by extensive, peer reviewed research and hard-won scientific scholarship. This approach stands in stark contrast to Barrett’s own fast and loose, “things don’t need to be tested [because] they simply don’t make any sense” methodology.
I couldn’t help but laugh at the characterization here. A better way to put it would be, just as quacks, antivaxers, and pseudoscientists do, Null cherry picks the literature to find dubious studies that support his preconceived notions.
As for that last part about “things don’t need to be tested [because] they simply don’t make any sense,” that’s clearly a straw man version of the main precept of science-based medicine. Basically, Null mischaracterizes the SBM view about, for example, homeopathy as being that it doesn’t warrant further scientific testing because it “doesn’t make sense.” While that is the argument—although it would have been more accurate to say, “doesn’t make sense scientifically“—it is an intentionally incomplete statement of the SBM argument that makes it sound as though skeptics dismiss pseudoscience like homeopathy out of hand just because it “doesn’t make sense.” In actuality, the correct argument is that homeopathy’s precepts, particularly the law of infinitesimals, which states that diluting a homeopathic remedy makes it stronger, violate multiple longstanding well-supported laws and theories of science. As I like to say, for homeopathy to “work,” not only would much of what we know about physics, chemistry, and biology have to be wrong, it would have to be spectacularly wrong. As a consequence, the prior probability, from a Bayesian perspective, that homeopathy can work, is so small as to be, well, homeopathic. The burden of evidence thus falls on homeopaths and believers in homeopathy (like, as you will soon see, Gary Null) to provide evidence from the basic sciences that show that homeopathy could actually work, something they as yet have failed to do.
I also couldn’t help but laugh out loud when I read the conclusion:
I can conclude that with absolute proof of an outstanding educational background, and his extensive clinical experience, that Gary Null is being attacked for what he represents: a viable challenge to the existing medical paradigm.
You know, that last line makes me wonder whether Gary Null actually wrote this article. It doesn’t matter. He wouldn’t have published it if he didn’t support its conclusions.
Gary Null vs. Susan Gerbic
More recently, Null has been very unhappy about Wikipedia, a taste of which I provided above from his lawyer’s letter to the Wikimedia Foundation. The result has been a two part screed, Wikipedia: Our New Technological McCarthyism, Part 1 and Part 2. It’s primarily in Part 2 where he lashes out at Gerbic.
It’s not at all surprising that Null would not appreciate Gerbic’s efforts, given that she’s created an organization, Guerilla Skepticism on Wikipedia (GSoW) dedicated to making sure that Wikipedia articles on topics with which skeptics concern themselves are scientifically accurate and that promoters of pseudoscience and quackery don’t manage to bias Wikipedia articles in favor of nonsense. Her group filled a definite need, and she answered the call. Indeed, the slogan of GSoW is, “The mission of the Guerrilla Skepticism editing team is to improve skeptical content of Wikipedia. We do this by improving pages of our skeptic spokespeople, providing noteworthy citations, and removing the unsourced claims from paranormal and pseudoscientific pages. Why? Because evidence is cool. We train – We mentor – Join us.”
Of course, this is exactly the sort of thing that quacks cannot abide, which is why they’ve attacked Gerbic before. For instance, five years ago, Deepak Chopra was very upset at the GSoW treatment of woo-meister Rupert Sheldrake. As I pointed out at the time, there is indeed a movement afoot to correct scientific misinformation and try to keep the entries in Wikipedia scientifically accurate. Chopra complained about this as though it were a bad thing, which to him it was, because Susan’s activism keeps entries that have anything to do with the sorts of quackery and pseudoscience he believes in from being transformed by quacks to glorify and promote those selfsame forms of quackery and pseudoscience—or at least from presenting them as though they were true or even just as scientifically valid points of view.
So what does Null think of Gerbic and her activities? He’s not a fan:
One group that has received Wikipedia’s full support and swallowed Dawkin’s “militant atheism” whole with steroids is Guerrilla Skepticism on Wikipedia (GSoW), founded by Susan Gerbic. GSoW actively seeks out and trains recruits to serve as an army of a Skeptic editors to wage wiki wars against those who research or advocate alternative medical treatment modalities. These are the most active of Wikipedia’s independent advocates editing alternative medicine content and pages critical of conventional drugs.
I can’t help but point out here that there was a previous version of this article in which this paragraph read a bit differently:
One group that has received Wikipedia’s full support and swallowed Dawkin’s “militant atheism” whole with steroids is Guerrilla Skepticism on Wikipedia (GSoW). Founded by a woman who owns a small portrait gallery in Monterrey, Susan Gerbic is a close friend of SBM guru David Gorski who fully endorses her organization’s internet militancy. GSoW actively seeks out and trains recruits to serve as an army of a Skeptic editors to wage wiki wars against those who research or advocate alternative medical treatment modalities. These are the most active of Wikipedia’s independent advocates editing alternative medicine content and pages critical of conventional drugs. To further proselytize her extreme Skepticism globally, Gerbic later founded the World Wikipedia Project to reproduce her successes on foreign language Wikipedias.
Why was the change made? I can’t know for sure, but about a week and a half ago my not-so-super-secret other blog noted many errors of fact in Null’s articles. As a result, the articles mysteriously disappeared for a couple of days, and when they reappeared they had been…edited. Also, Null appears to have removed several mentions of my name, perhaps because he was embarrassed, perhaps because he was ticked off. For instance, in the original version of Part 1, Null claimed that I was based at Wake Forest University. Truly, I take what I view to be as deserved credit for “helping” Null improve his accuracy, although I will likely point out more of his errors of fact before this post is done. For instance, one error that didn’t get corrected was that the World Wikipedia Project only existed from 2012-2014 and was merged into GSoW.
Elsewhere:
On her personal Wikipedia biographical entry, Gerbic is quoted as saying, “We rewrite Wikipedia, and proof the pages, we remove citations that are not noteworthy, we add citations, we do just about everything in Wikipedia to improve content.”[13] Of course, the majority of their “notable citations” reference back to Skeptic and SBM sources, such as Gorski’s ScienceBasedMedicine blog. “Improvements” are solely aligned with Skepticism’s doctrine. Gerbic’s other organization Skeptic Action is another stealth guerilla operation to disseminate cyber tasks for Skeptic trolls on Twitter, Facebook and Google+ to rapidly rate pages such as books listed on Amazon that question vaccination, homeopathy, and natural cancer treatments. Skeptic Action also utilizes a community drive system, which enables members to receive rapid alerts to rebut content posted on the internet.
Gee, Null says that as though it were a bad thing. That’s not surprising, because Wikipedia would be a great marketing tool for quacks, and a team of skeptics preventing pseudoscience and dubious references from being inserted into Wikipedia entries is a major buzzkill when it comes to shiny happy propaganda about alternative medicine. Indeed, Null even inadvertently provides an example:
In this particular case, Debby Vajda, President for the Association for Comprehensive Energy Psychology (ACEP), provided 51 peer-reviewed articles and studies, 18 which were randomized controlled studies, appearing in professional journals, including the American Psychological Association, the Journal of Clinical Psychology, the Journal of Nervous and Mental Diseases, Psychotherapy Theory Research and Practice and others showing positive statistical results outside the range of chance. She commented on Change.org, “Every edit to the energy psychology Wikipedia page that attempts to reference findings from these well-respected, scientific journals is summarily deleted… The American Psychological Association does not think we are ‘lunatic charlatans.’ Neither does the Association of Social Work Boards, the National Board of Certified Counselors, or the National Association of Alcohol and Drug Abuse Counselors, all of which approve ACEP to provide continuing education to their professional members for the study of energy psychology. The Wikipedia page is out of step with existing peer-reviewed research on this topic, and opinionated, self-described “skeptic” editors are resisting any change.”[18]
Apparently the scientific evidence was insufficient to pass Wikipedia’s administration review. The page still defines Energy Medicine as a “pseudo-scientific belief.”[19]
See? Energy medicine “quacks” are trying to promote their quackery with the usual dubious pseudoscientific studies. Fortunately, Wikipedia is quite correct to define energy medicine as pseudoscience. As I like to say about reiki, for instance, it’s faith healing substituting Eastern mystical religious beliefs for the Christian beliefs usually used to justify faith healing. A list of the sorts of articles ACEP promotes as “rigorous” include the usual nonsense about Emotional Freedom Technique, acupuncture, thought field therapy, “Wholistic Hybrid Derived from Eye Movement Desensitization and Reprocessing and Emotional Freedom Technique,” and the like. (EFT, which is pure wishful thinking quackery, seems to be very big among this group.)
Null also blames it all on “militant atheism”:
The Center for Inquiry (CFI), the umbrella organization that serves as the mother chapel for the Skeptic movement, fully embraces Dawkins’ atheistic Scientism. In 2016, the Richard Dawkins Foundation for Reason and Science merged with CFI. Its stated mission is to “foster a secular society based upon reason, science, freedom of inquiry and humanist values.”[12] Laudable words, but the Center fails horribly to tolerate, let alone respect, the freedom of others to their beliefs and the freedom to choose a medical intervention of their choice. Any discipline of inquiry that is performed outsides the Center’s narrow interpretation of science is condemned as heresy, exposed and publicly maligned. Everything that deals with religion and spirituality, the paranormal, unexplained phenomena, and alternative and natural medical modalities are accused of con-artistry. Other leading major Skeptic groups are the Committee for Skeptical Inquiry, the Council of Secular Humanism, the James Randi Educational Foundation and the SBM-related Commission for Scientific Medicine and Mental Health.
The latter publishes the Scientific Review of Alternative Medicine, founded by Skeptics at Stanford University and the Committee for Scientific Investigation of Claims of the Paranormal. The publication makes the narcissistic claim of being the only journal that properly analyses alternative medical claims.
I do love how out of date Null’s information is. For instance, the James Randi Educational Foundation has basically gone into hibernation. Sure, it still exists, and supposedly has converted itself into a grant-making foundation, but I haven’t heard much of anything about such grants being awarded since the JREF made that change. Indeed, its website appears not to have changed much two and a half years, the sole recent announcement since 2015 being in January, in which JREF announced that Susan Gerbic received the 2017 award from the JREF, saying that the award “is given to the person or organization that best represents the spirit of the foundation by encouraging critical questions and seeking unbiased, fact-based answers.” Susan confirmed this award to me, and it cheered me up a bit to learn that JREF isn’t completely moribund.
As for the rest, does Null not know that the Commission for Scientific Medicine and Mental Health unfortunately has been moribund for years. The most recent articles on its website are from 2005, for cryin’ out loud! The most recent issue of Scientific Review of Mental Health Practice (SRMHP) was published in 2007. The most recent issue of the Scientific Review of Alternative Medicine (SRAM) was published in 2002. It’s sad, yes, very sad, and I do know that every now and then someone brings up the idea of trying to resurrect SRAM and SRMHP, but in reality they are probably no longer necessary given the rise of blogs and online publications. That doesn’t stop Null from attacking them as though they were still relevant.
Gary Null versus SBM on behalf of homeopathy
If there’s one thing that seems to tick off Gary Null, it seems to be the opposition to homeopathy by supporters of science. He mentions it on multiple occasions in his “McCarthyism” articles and most recently published an article co-authored with Richard Gale (the Executive Producer of the PRN.FM) entitled Slaying Homeopathy and Americans’ Freedom of Health Choices. It begins, as many of Null’s articles begin, with an attack on the medical profession as being against—of course!—freedom:
A fundamental foundation of American democracy is our freedom of choice. We have the legitimate right to pursue a career, decide where to live, what to read or watch, the freedom to vote or not, and decide who to marry or live with. Americans hold this these freedoms sacred, at least in theory.
However, during the past decades, an ethos has arisen, associated with the American Medical Association, the pharmaceutical industry, professional organizations, medical publications, media health reporters and legislators largely benefitting from the largess of private drug companies adamantly pushing to limit our medical choices. This medical regime wants to choose and dictate the types of medicines available and enforce how we are diagnosed for mental conditions. They have mastered the art of pathologizing life. We have little choice in having access to public water without fluoridation or the right to vaccinate our children or not. To illustrate this point, the Department of Health and Human Services (HSS) and FDA curtailing all positive information about homeopathy and making efforts to eliminate it altogether. This is accomplished under the misinformed ruse that it is unproven and has not undergone the rigor of conventional medical science. Therefore, homeopathy should be banned.
Yes, it is true that the FDA and FTC have of late made promising moves towards regulating homeopathy and its advertising just like other over-the-counter drugs. Yes, to those of us supporting SBM, this is a good thing, although I personally think that the FDA and FTC haven’t gone far enough.
Naturally, Null doesn’t like skeptics:
If we listen solely to the Skeptics – a quasi-rationalist movement of scientific idolatry — and the advocates of Science-Based Medicine (SBM), and now increasingly government health officials, homeopathy’s explanation is always the same. Homeopathy is only water. It is foolish and unscientific. Worse, we are told it is dangerous and even life-threatening. Homeopathy, so the Skeptics believe, fails to meet the standards of modern evidence-based medicine and should therefore be avoided. That is the end of the story.
Would that it were true that government health officials are increasingly viewing homeopathy as The One Quackery To Rule Them All, which it is! Sadly, despite promising moves here in the US and across the pond in the UK, in a lot of other countries homeopathy is almost as accepted as real medicine. For instance, recently in France a group of physicians has been harassed with complaints to French medical authorities for daring to speak out against the government’s support of homeopathy. Meanwhile, Agnès Buzyn, ministre des Solidarités et de la Santé (Minister of Solidarity and Health), stated her support for homeopathy, specifically, “If it continues to be beneficial, without being harmful, it will continue to be reimbursed.” This is a pretty damned irresponsible thing for any minister of health in any country to say. Don’t believe me? Check this out:
“Si ça continue à être bénéfique, sans être nocif, ça continuera à être remboursé”➡ @agnesbuzyn ministre des Solidarités et de la Santé#BourdinDirect #Santé #homeopathie pic.twitter.com/OsfsmbG5yP
— RMC (@RMCinfo) April 12, 2018
In Germany, homeopathy remains popular as well. Ditto India and a lot of other countries. Even though homeopathy is no longer supported by the NHS the way it was, the royal family still uses it, and Prince Charles is its champion. Indeed, Null uses the popularity of homeopathy in other countries to claim that it must have some validity.
Predictably next up, likening SBM to a religion:
The priests in this Citadel who serve the scientific and technological elite of postmodernity, namely federal health agencies and the pharmaceutical industrial complex, in fact properly speak their truths about homeopathy and other non-conventional medicine. But their truths only exist within a very specified, narrow, and limited context or field from which they criticize science they either don’t or refuse to understand or are incapable of accurately explaining. For this reason they are easy to criticize because in fact they are the self-anointed spokespersons for the dire failings of modern scientific culture, conventional medicine and repressive government health policies.
Dr. Steven Novella, the acknowledged founder of Science-Based Medicine describes homeopathy as;
“a prescientific philosophy-based system on magical thinking…. Essentially, homeopathy uses fanciful treatments that are based on silly ideas, such as the personality of the patient, but also “sympathetic magic.” The belief is that homeopathic remedies contain the magical essence of symptoms and can be used to cure those same symptoms.”[3]
Another spokesperson for the SBM movement is oncologist and breast cancer surgeon Dr. David Gorski (aka ORAC). Gorski writes, “Homeopathy is the perfect quackery. The reason that homeopathy is so perfect as a form of quackery is because it is quite literally nothing.”[4] But are the Skeptics perhaps accurate in their assessment about homeopathy? Is over-diluting a substance in water whereby the atoms are no longer detectable any more bioactive than plain water? Yes, we need to ask these questions. But it is also wise to parallel this inquiry and ask whether most conventional drug therapies safely work and properly improve and save lives.
Yes, these are all familiar tropes. Medicine is a religion, and, oh, by the way, conventional medicine doesn’t work. After this follows a litany of all the clinical trials of homeopathy that have “proven” that homeopathy is safe and effective. Naturally, none of them are convincing if you really look at them with a skeptical eye. How often have we here dissected various clinical trials of homeopathy and shown that they do not show what adherents think they show? Null also ignores that one has to examine the totality of the scientific evidence, basic, preclinical, and clinical. After all, by their very nature, clinical trials occasionally produce false positive results, which means that homeopaths and their supporters (like Null) can always manage to cherry pick some studies with weak statistically significant results.
Elsewhere, in part 2, Null actually claims that skeptics do not understand homeopathy. Indeed, he views homeopathy as an example of what’s wrong with skepticism:
Homeopathy is an excellent example of Skepticism’s unsound and frequently unsubstantiated criticisms. Simply because SBM physicians may not understand biophysics, quantum energy, and physical properties of water should not close the door on homeopathy as mere quackery. Surely Skeptics will embrace the value of nanotechnology without understanding the physics of spatial quantum confinement behind it. Nanomedince is rapidly becoming part of conventional medicine’s drug arsenals. Safety studies for nano-drugs are weak at best. Yet there are analogous features to nanotechnology and homeopathic theory in terms of spatial physics and force. Furthermore, in Europe, homeopathy is a preferred alternative treatment modality among doctors. In India, where it is most popular, 62% of homeopathic users have never tried conventional drugs, and 82% of those in an AC Nielsen survey said they would not switch to allopathic treatments. In France, 94% of surveyed pharmacists acknowledged they recommend pregnant women to use homeopathic remedies instead of pharmaceutical drugs. Homeopathy is also taught in 21 of 24 French pharmacology schools. Seventy percent of French physicians approve of the discipline.[25]
And, in Slaying Homeopathy:
We have identified several possible reasons. First, American physicians, and notably the Skeptics, who oppose homeopathy and other scientifically proven alternative medical modalities, are not intellectually capable of understanding the principles of homeopathy and hormesis. Skeptics are victims of their own “strange loops” — a term coined by Douglas Hofstadter, the Pulitzer Prize winning professor of cognitive science and mathematics.
“Strange loops” can account for the paradoxical “irrational rationality” so prevalent in Skeptics’ faulty reason when ranting in their comic diatribes against homeopathy, acupuncture, CAM, etc. Skeptics are victims of scientific ignorance and exist within a local tribal tunnel-reality of their own creation.
Second, and alternatively, SBM advocates are fully aware of the scientific literature supporting homeopathy but have been institutionally conditioned to be biased against it. And third, they are fully aware but homeopathy competes with their own preferred conventional therapies. The first possibility is undoubtedly true for Skeptic organizations, such as the Committee for Skeptical Inquiry, and its advocates. The second and third rationales are likely the case for the medical professionals who follow Science-Based Medicine, and Skeptics like Craig Pearcey who have a very sound grasp of many of homeopathy’s underlying principles as well as quantum mechanics. Allowing SBM’s spokespersons the benefit of the doubt that their blogs and guerrilla tactics are not directly receiving funds from Big Pharma and other private or professional medical interests, nevertheless they all summarily choose to deny any scientific evidence to support homeopathy’s value.
Shout out to Craig! I also “thank” Null for admitting the possibility that we SBM skeptics aren’t all pharma shills, although one can’t help but note that he certainly implies it while “giving us the benefit of the doubt.”
Actually, it is because SBM-supporting physicians do have an understanding of biophysics, quantum energy, and physical properties of water that we reject homeopathy as pseudoscience. It is because quacks like Null do not understand these things and are unduly impressed by the shiny, happy pseudoscience and mysticism invoking quantum physics as though it were magic. Also, one more time, homeopathy has nothing to do with nanoparticles, and appealing to homeopathy’s popularity is meaningless. You can be sure that Null won’t mention that homeopathy is big business in France (Boiron, anybody) and that powerful forces actually suppress criticism of homeopathy by SBM physicians. Yes, in France, criticizing homeopathy too vigorously can land you in a world of hurt, including complaints against you and even lawsuit, and the Minister of Solidarity and Health fully supports homeopathy. In France, homeopathy is mainstream. To SBM, that’s a problem. To Null, it’s a good thing.
The Null hypothesis
Although his star has been eclipsed by younger interlopers such as Mike Adams, having been at his chosen profession for well over 40 years now, Gary Null nonetheless remains influential among proponents of quackery, even though his methods are very old school (articles, radio shows and podcasts, and movies) and his voice, whenever I dare try to listen to his radio show, makes an excellent nonpharmacological insomnia aid, which no doubt Null would approve of over something like Ambien. For as long as I’ve paid attention to him (about a decade now), his M.O. has been basically the same and consists mainly of:
    • Attacking conventional medicine as dangerous and unscientific.
    • Cherry picking bad studies of alternative medicine to
    • Crying persecution by government and the medical profession due to his preferred quackery representing a “viable” alternative to medicine.
    • Conspiracy mongering.
    • Ad hominem attacks against his critics as being in the thrall of big pharma or ideologically biased against alternative medicine.
    • Portraying conventional medicine as a religion.
Which brings us to Null’s conclusions.
From part 2 of his McCarthyism article:
Wikipedia is embedded skeptic groups that attack those who would tell us the truth, the guardians of the social media galaxy. We are brainwashed 24-7 without warning. No trepidation. No open debate. We are solely passive consumers in the wiki matrix. Objectivists, as The Economist article notes, functions best when social conditions reinforce a bee-hive mentality. This is what enables Skeptic leaders to cling to their perceptions of intellectual superiority. In the meantime we have a compliant nation, a population obedient and only buying.
Project much, Mr. Null? The people who would most benefit from a compliant nation, a population obedient and only buying, would be Gary Null, Mike Adams, and his fellow quacks. Certainly they’d benefit far more than any SBM proponent would. We have our pesky dedication to science that keeps us from taking full advantage of the free market. That’s why financially we’re always at a disadvantage—and not by a little. Null can never admit that, though, because his very business model depends on him portraying himself as an underdog fighting the system for Truth, Justice, and the American Way:
To win the war on medical truth, Skepticism needs to be called out for what it genuinely is, an aberrant form of destructive skepticism masquerading as the arbiters of authentic science. Only by exposing these individuals and organizations, the flaws in their “irrational rationalities,” and the regressive system of healthcare they represent that threatens the well-being of the public, will homeopathy have a chance of gaining the wider acceptance it deserves.
Say the duo who claim that skepticism is a “curious mix of Orwellian fascism and a quirky technological totalitarianism” and the “darker side of Liberalism, with noticeable parallels to Ayn Rand’s Objectivist and autocratic absolutism.” Yes, I do think that they project.
Of course, in the end, the “freedom” that Null and Gale espouse is freedom from the shackles of the pesky science that demonstrates homeopathy (as well as the other varieties of quackery that they both support) to be ineffective. The reason Null and Gale attack Wikipedia and skeptics is not out of some love of The Truth, but because we get in the way of selling their products.

Gary Null’s Attack on SBM
Gary Null’s fact- and logic-free attack on SBM.
Steven Novella on August 7, 2019
Link
Gary Null is a long time promoter and marketer of alternative nonsense. He promotes homeopathy, alternative cancer cures, sells supplements on his website, has promoted HIV denialism and the notion that vaccines cause autism, and is anti-GMO. He also really doesn’t like Science-Based Medicine (SBM) and skeptics in general. Of course he doesn’t – because we call him out on his pseudoscience.
Many of the charlatans we expose simply ignore us and continue to make their millions, but some, like Null, have chosen to go on the attack. He has done this before, nicely taken down by David Gorski. I suppose it’s my turn to respond to the latest hit piece by Null.
And make no mistake, this is a hit piece. In it Null accuses us, by name, multiple times of being mentally ill, of being part of a cult, of having conflicts of interest, of being a front group, and being apologists for industry. He also attacks the skeptical movement in general, completely misinterpreting internal debates about what the scope and focus of the skeptical movement should be. But mostly he harps on the idea that we are narrow-minded “nihilists”. The article can be charitably called a rant, and in it is a Gish-gallop of misdirection and misconceptions. I cannot address everything he gets wrong (that would take a rather long series of articles) so I will focus on his more philosophical points about skepticism and SBM itself.
He portrays modern skepticism this way:
Although Skepticism has been a worldview dating back to the nineteenth century, today’s Skepticism is far more radicalized. Because Skeptics believe they represent the pinnacle of scientific materialism, many of the movement’s celebrity gurus feel they are the entitled saviors to redeem modern civilization from thousands of years of past history when human societies utilized medicinal plants and ancient mind-body practices to treat illnesses and the role of religion, spiritual practices, faith and belief to promote a sense of well being and psychological wholeness. As with so much of our dysfunctional postmodern world, Skepticism is therefore a natural outgrowth of white-dominant, patriarchal entitlement that continues to plague postmodern society. It is another perversion of identity politics however disguised under the banner of science.
Not quite. Actually I personally, and SBM collectively, represent a branch of activist skepticism that explicitly does not oppose religion per se, or faith or spirituality. We simply feel that faith should be properly treated as a personal choice, and not imposed on others or on society. We also feel that the medical profession should be based on science, and not on faith or spirituality. Faith is too personal and subjective to form a basis of medicine, and professional ethics argue strongly against health professionals imposing their faith on patients. The nature of science is such that we can use objective evidence and logic to form a consensus. The same is not true of faith.
We also may oppose bad choices, such as parents withholding basic medical care from a sick child, that are justified by faith (but regardless of the justification). We don’t attack the parents’ faith, but rather their imposition of that faith on a sick child that should be their responsibility to care for. It seems like this is all far too much nuance for Null, or more simply he is motivated to attack his simplistic straw men of our positions rather than truly understanding them and treating them with even minimal fairness.
He finishes that opening paragraph with a flourish about “white-dominant, patriarchal entitlement” while simultaneously accusing us of “identity politics”. This is a common false accusation by pseudoscientists – trying to portray science itself as a white male institution, therefore defending science is defending white patriarchy. Of course this stands in stark contrast to the reality that science is perhaps the only institution that is based on principles that are explicitly open, objective, transparent, and universal. There is no “white” science or “Western” science – there is just science. Of course historically science has been abused, like every institution has been abused, to serve those in power. That is an indictment of our societal ills, not science itself. But again, it’s easier to confuse than to enlighten.
And now we come to the meat:
In principle, Science Based Medicine largely supports Evidence Based Medicine. And to their credit, SBM’s foremost spokespersons Steven Novella and David Gorski opening criticize EBM’s shortcomings, including the now epidemic of erroneous research being published in medical journals, the increasing trends in professional bias in order to reach positive results, and journals’ financial incentives to publish junk studies. However, considering EBM’s flaws and failures, SBM perceives itself as the next great leap for modern medicine in order to establish scientific consensus on medical discovery and therapeutic practices by including the plausibility principle. Repeatedly, without any sound understanding for why a certain alternative health therapy either succeeds or fails, Skeptics invoke plausibility as the only necessary criteria to discard outright non-conventional practices and therefore to advocate against funding research to investigate any promises they may hold.
Null, it seems, feels he has to acknowledge that SBM criticizes mainstream scientific medical practice, because we so clearly and often do. In fact, the charlatans like when we do this because they can twist our healthy criticism into a nihilistic denial. He doesn’t realize, or want to admit, however, that this simple fact contradicts pretty much the rest of his screed against SBM as being a tool of that same mainstream institution. He simply ignores the glaring self-contradiction he has set up. If we are the tool he says we are, why would we spend so much time nitpicking about p-hacking, Bayesian analysis, and medical journal editorial policy?
In fact, further down he says:
If SBM would put more attention on the serious health risks of just a single common over the counter drug, acetaminophen or Tylenol, imagine how many people it would save compared to its fear mongering about supplements such as Vitamin C and Omega-3 fatty acids.
This was a poor choice of an example. Apparently Null was too lazy to do a Google search, for if he did he would have found a 2014 article by Harriet Hall: “Tylenol May Not Be As Safe and Effective As We Thought“. Oops. This was not a huge coincidence befalling Null. We regularly take on common misconceptions, whether they deal with mainstream medicine or so-called alternative medicine. Just within the last week Clay Jones wrote an article entitled “Think Twice Before Giving Young Children Reflux Medications“.
But of course we spend more time addressing fringe claims. That is our niche. There are plenty of mainstream medical outlets dealing with conventional medicine. There is a paucity of hard scientific examinations of fringe medicine, and that is a role we very deliberately fill.
But that is only part of our mission. Our core mission is to promote the process of SBM in all of health care. Of course the pushers of fringe and bizarre treatments don’t like that, because when you shine the bright light of science on their claims they don’t do well. But let’s get back to the plausibility principle, that Null (and others) like to attack. Again, Null misses all the nuance, despite the fact that we have written extensively on this topic and provided all the nuance for him. So let me do it again.
Of course proponents of fringe treatments hate the concept of plausibility, because their treatments are often highly implausible. That would love nothing else than to discard the idea entirely, and that, of course, was EBMs unintended mistake. So they portray the concept of plausibility as nothing more than bias and narrow-mindedness. But this is patently not true.
First, I think we have made a strong case that you have to consider prior plausibility when evaluating the clinical evidence. Clinical evidence is often not definitive, and so we have to put it into the context of plausibility. Moderate clinical evidence for a highly plausible treatment is not the same for moderate clinical evidence for a highly scientifically implausible treatment. Even from a purely statistical point of view, the probability of the former treatment working is far greater than the latter.
Further, we simply advocate a holistic (if you will) approach to the evidence. We suggest we should be looking at all the scientific evidence systematically, and put that evidence into the clearest scientific context possible. We have to consider the relationship between rigor and outcome, replicability, evidence for p-hacking, publication bias, and citation bias, take into account the decline effect in research, and compare clinical outcomes to overall plausibility. We do not, as Null would like you to believe, use plausibility alone as the sole reason to reject a new hypothesis.
Further, we do not require an understanding of mechanism of action to accept a treatment (although it helps put the evidence into context). It’s possible for clear-cut home-run clinical evidence (multiple independent large rigorous clinical trials showing a clear and consistent result that is statistically and clinically significant) to exist in the absence of known plausibility. In such a case, scientists would then try to reverse engineer how such a treatment works in order to fill in clear gaps in our knowledge.
But what we often have with the types of fringe treatments that Null champions, like homeopathy, is very low scientific plausibility combined with low quality studies, or clinical studies that are generally negative and never reach a reasonable threshold of acceptance. With homeopathy, for example, even if you ignore prior probability, the clinical evidence is simply negative.
Null then attacks us for saying that further clinical research is unnecessary. He misrepresents what is my explicit position. It’s OK to invest in research in highly unlikely treatments, but our research dollars should be proportionate to plausibility. This is not just my position – this is the position of the NIH and most funding agencies. You have to show there is a reason to think that your intervention is safe and effective before they will give you money.
Further, there have been over 1,800 studies in homeopathy. Homeopaths have had two centuries and hundreds of studies to demonstrate whether homeopathy works, and they have failed to do so. The simplest explanation for this is that homeopathy doesn’t work, which is further consistent with its low plausibility. I don’t see how you can make the case for doing still more clinical research into this dead hypothesis. At some point it becomes unethical to subject people to research on failed treatments.
Again, this is all nuance (but clearly stated nuance) that Null ignores in order to create his straw man that we reject ideas we don’t like out of hand.
You may notice I am just a couple of paragraphs in, and already this is one of my longer articles. The distortions and false claims keep coming from Null, but hopefully this will give you an idea of the intellectual value of his arguments (which are homeopathic).

Joe Biden defeated Donald Trump. Now what for science-based medicine?
In 2016 and 2020, scientists expressed surprise and alarm at the results of the Presidential election. In 2016 it was alarm that someone as antiscience as Donald Trump was elected, and in 2020 it was over how close the election was, given Trump’s dismal record on science, medicine, and the COVID-19 pandemic. Are scientists out of touch? And now what, for federal science policy and the response to the COVID-19 pandemic?
David Gorski on November 9, 2020
Link

On Saturday, after four days of ballot counting, tabulation, speculation, commiseration, and angst on all sides, it became clear that Joe Biden had won Pennsylvania, and with it the necessary Electoral College votes to put him over the top for the Presidency. True, there will be recounts and legal challenges, but most agree that President Trump doesn’t have much of a leg to stand on legally. It is also unlikely that recounts will change the results in enough states to flip the election back to President Trump. What this means is that, barring a truly unexpected development, on January 20, Joe Biden will take the oath of office and become the 46th President of the United States.
I’m not interested in discussing the politics, mechanisms, or machinations of the election itself so much, though. We at SBM generally strive to remain nonpartisan, although we’ve never been apolitical, given our support for science-based policy, such as cracking down on quackery, not legitimizing homeopathy with FDA approval, and supporting strong school vaccine mandates. That being said, this election season has been very painful to watch and participate in (at least for me), and, I’m sure, nerve-wracking for some of my friends in the UK, Europe, Australia, and other countries around the world to watch from afar. Regardless of party, what I am interested in is what the new administration will mean for science and, as in the name of this blog, science-based medicine in terms of federal law and policy. Unfortunately, these days it is impossible not to be at least somewhat partisan, given the policies of the two parties with respect to science in 2020.
In any event, I can’t help but note that I did the same thing four years ago, commenting on what I thought might be the consequences after Donald Trump had first become President-Elect. So did Jan Bellamy. Interestingly, many of our predictions at the time were fairly prescient, although none of us can take that much credit for brilliance in making them given how much the Trump campaign had telegraphed its hostility to climate science, environmental science, and what it perceived as excessive “regulation” by the Food and Drug Administration (FDA) over drug approval and development. It didn’t take much knowledge to produce at least a rough outline of what would be coming.
Scientists’ reactions to 2016 versus real-life outcomes
In 2016, Jann Bellamy noted that Trump’s victory had “stunned scientists“, citing a number of them saying things like:
I do breast cancer research for my PhD….Scared not only my future but for the future of research and next years @NIH budget😱.
— Sarah R. Hengel PhD (@science_SRH) November 9, 2016
this is terrifying for science, research, education, and the future of our planet. I guess it's time for me to go back to Europe
— María Escudero Escribano (@MariaEscEsc) November 9, 2016
good luck w funding.
ps my heart is *breaking* for all the young scientists I helped to teach and raise.
— michele ly (@themichele) November 9, 2016
Restrictions in funding opportunities, more scrutiny on research questions, no funding for climate change #Election2016
— Becky Carmichael (@bcarmi1) November 9, 2016
i want to die right now so that's one fewer western blot for the world
— Kevin Ng (@KevinWNg) November 9, 2016
Also:
“With a climate skeptic as President and a creationist as vice-president, scientists can only be worried.” —Albert Descoteaux, a biologist studying host-parasite reactions at McGill University in Montreal, Canada.
And:
“Trump will be the first anti-science president we have ever had,” Michael Lubell, director of public affairs for the American Physical Society in Washington, D.C., told the journal Nature. “The consequences are going to be very, very severe.”
Perhaps the most telling reaction cited was this one:
“I am confused, angry, depressed. I feel much like I do when I receive the comments on a rejected paper that the reviewers have torn apart. This morning I realized that I don’t actually know a Trump supporter who I could talk to about the election. How can I reach the public if I’m only speaking to my own circle?” —Peter Peregrine, an anthropologist at Lawrence University in Wisconsin and the Santa Fe Institute in New Mexico.
Therein lies part of our problem as scientists. I’ll have more to say about that near the end of this post, but for now I want to start by mentioning that my reaction to the election of Donald Trump as President focused primarily on its potential effect on science policy with respect to SBM. (Surprise! Surprise! I’m a surgeon, translational scientist, and I edit a blog called “Science-Based Medicine”! SBM and its opposites are what I’m most interested in.) For example, my first SBM post after the 2016 election concentrated on Trump’s (and, let’s face it, the Republican Party’s) obsession with “deregulation” and its definite sympathy towards the “health freedom movement” and what that might mean for “right-to-try,” given that Trump would be entering office with control of both Houses of Congress as well. It turns out that, in retrospect, I was correct about a lot of things, and a federal “right-to-try” law was passed in 2018 (although it didn’t really do much of anything, fortunately). I was also correct about the push to loosen drug approval standards at the FDA, which began immediately, when Trump floated the names of Peter Thiel-backed libertarians to head the FDA, one of whom thought that the free market should decide which drugs are effective and that the FDA should only make sure they are safe before approving them, the other of whom thought that online rating systems, an “Uber” or “Yelp” for drugs if you will, would be a fine idea to evaluate the safety and efficacy of medications. Scott Gottlieb, who was ultimately chosen, turned out to be the “least bad” choice in that he at least was competent. However, he was a genuine “pharma shill” with deep ties to the pharmaceutical industry who believed in “streamlining” the FDA drug approval process further than it had been “streamlined” already, even though arguably the process had already become too lax.
Meanwhile, Trump chose Dr. Tom Price, a member of the medical crank organization disguised as a medical professional society (the Association of American Physicians and Surgeons, or AAPS) to be the Secretary of the Department of Health and Human Services (HHS). Recall that AAPS has long promoted a wide range of conspiracy theories, including antivaccine pseudoscience (including Andrew Wakefield), and has since pivoted to promoting COVID-19 misinformation and conspiracy theories. This reminds me, the Trump administration was already quite awful with respect to federal policy relating to SBM even before the COVID-19 pandemic. Every year, it proposed slashing the NIH budget starting immediately upon taking office, for example. Fortunately, every year, even the Republican-controlled Congress didn’t oblige him and even significantly increased the NIH budget. In addition, his first CDC Director, although refreshingly very pro-vaccine, had a history of selling quackery, while there were reported attempts to censor the CDC’s budget requests by flagging it for words that should not be said.
COVID-19 intervenes
Regular readers will likely have noticed that over the last several months the COVID-19 pandemic has come to dominate the blogging here at SBM. This is no surprise, of course, given that the pandemic, our responses to it in terms of policy, the search for treatments and a vaccine, and the political response to federal, state, and local policies represent the single largest ongoing story related to medicine in 2020 and will without a doubt continue to be so well into next year, if not for the full year. How could it be otherwise? Basically, the COVID-19 pandemic is not going away any time soon, the wishful thinking of President Trump notwithstanding, and the US has had disproportionately more cases and more fatalities than any other nation on earth. That last part is just a fact. The US represents only 5% of the world’s population but accounts for nearly one-fifth of the world’s deaths from COVID-19, roughly 237,000 so far as of the time I write this, and we are racking up over 100,000 new cases per day. By any reasonable measure, things are not going well with respect to COVID-19 in the US. As a nation, we are not handling the pandemic well.
There are many reasons for this, not the least of which is how COVID-19 has unleashed conspiracy theories all over the world, but in particular in the US. The antivaccine movement rapidly joined forces with COVID-19 conspiracy theorists, minimizers, deniers, and cranks, an unholy alliance that developed very early in the pandemic, to become a major force in resisting public health interventions to slow the spread of COVID-19 and mitigate its massive harm. Antivaccine conspiracy theories merged with COVID-19 conspiracy theories and then with QAnon to produce a toxic soup of seemingly unstoppable misinformation and disinformation that millions, ensconced in their social media bubbles, have come to believe, while antivaxxers launched a pre-emptive disinformation campaign to cast doubt on the safety and effectiveness of COVID-19 vaccine candidates that had rapidly gone into development.
Here’s one area where the Trump Administration, in its hostility to regulation coupled with its desperation to find a quick fix for the pandemic, made things so much worse. First, it named its COVID-19 vaccine initiative “Operation Warp Speed,” a name that implied that speed was being valued above all, including safety, to the point where even pro-vaccine voices (including myself) expressed doubt over whether the FDA would adequately assess and confirm the safety of COVID-19 vaccine candidates before approving them. It took a lot to lead those of us as pro-vaccine as, for example, Paul Offit, Peter Hotez, and myself, to question whether the FDA would adequately assess the safety and efficacy of COVID-19 vaccine candidates, but damn if the Trump administration didn’t manage it.
Then there was the active undermining of science-based responses to the pandemic. Anthony Fauci, Deborah Birx, and others tried to emphasize responses to the pandemic based on the best public health science. True, the disaster that has been the federal response to the COVID-19 pandemic wasn’t all Trump’s fault. (In an event this large, it never is anyone’s fault, solely.) For example, there were missteps coming from the President’s coronavirus task force. Messaging on masks early in the pandemic was borderline disastrous, such as when the main message being promoted was that people probably shouldn’t wear masks routinely unless they were sick. This was likely adopted in an effort to preserve the supply of medical grade masks for frontline healthcare workers and based on the then-current understanding that COVID-19 probably wasn’t contagious in asymptomatic patients, a conclusion that turned out to be tragically wrong when it became clear that asymptomatic transmission of SARS-CoV-2, the coronavirus that causes the disease, is a major means of its spread.
Whatever the reasons in the “fog of war” early in the pandemic for this misstep, the mixed messaging on masks wasn’t entirely Trump’s fault. What was Trump’s fault was his decisions to refuse to wear a mask, to mock those who did wear masks (including his opponent former Vice President Joe Biden) as “weak,” and to politicize the wearing of masks such that not wearing one became a mark of Trump’s “tribe,” while wearing a mask marked a person as being a “liberal,” or the enemy “tribe.” This continued even after the evidence made it clear that masks work to slow the spread of COVID-19. Unlike Fauci, however, Trump refused to change his mind when evidence had by June become pretty definitive that masks work, encouraging the spread of disinformation about masks. (Also, masks don’t make you sicker.) Even after he himself had contracted COVID-19 six weeks ago, thanks to a lack of concern about most precautions other than testing in the White House, and became, I suspect, sicker than was being let on, President Trump continued to attack public health messaging designed to slow the spread of the disease and portrayed himself as having “beaten it”, all while continuing to portray the US as just “rounding the turn” with respect to the pandemic, despite the copious evidence otherwise.
And don’t even get me started on the other—shall we say?—unhelpful things that Trump did. Who could forget his promotion of hydroxychloroquine, an antimalarial and mildly immunosuppressive drug whose efficacy was at the time unproven, as a “cure” for COVID-19? This led to a run on the drug, resulting in shortages for those patients who needed it to treat their rheumatoid arthritis and other autoimmune diseases. Spurred by Trump, all manner of quacks (e.g., Drs. Mehmet Oz, Vladimir Zelenko, and Didier Raoult) and Trump administration officials (e.g., Peter Navarro) promoted the drug as a virtual “miracle cure,” complete with alternative-cancer-cure-like testimonials, and portrayed those casting doubt on its efficacy (or even just urging caution and waiting for results from randomized clinical trials) as downright evil people who wanted more people to die. All of this led to a premature emergency use authorization (EUA) by the FDA for hydroxychloroquine to treat COVID-19, an EUA that was rescinded not long after as randomized clinical trials failed to demonstrate efficacy. Even now, hydroxychloroquine is the Black Knight of COVID-19 treatments, its adherents refusing to admit what science has shown and astroturf campaigns still actively promoting the drug, along with the conspiracy theory that big pharma and other nefarious “deep state” forces had “suppressed” evidence that it works.
Meanwhile, the politicization of the FDA and CDC got so bad that last month the Government Accountability Office launched an investigation, charging that “the Trump Administration has reportedly pressured the CDC and FDA throughout the COVID-19 pandemic, repeatedly applying political pressure and imposing orders on career scientists that undermine the agencies’ credibility and independence”. This sort of pressure took the form of political pressure to approve a vaccine by EUA, pressure that led me to ask two months ago if we can even trust the CDC and FDA any more, given how the messaging from the CDC had been affected and the declining trust in the FDA’s drug approval processes, given the obvious political pressure brought to bear on the agency. In the last couple of months, even Anthony Fauci and Deborah Birx were sidelined, in favor of someone who is—let’s face it—a hack, a radiologist who hasn’t practiced in years and has no training or experience relevant to the pandemic in public health, epidemiology, or infectious disease, a Hoover Institution political flack named Dr. Scott Atlas. This is a man who has basically embraced the nonsolution based on likely unattainable “herd immunity” known as the Great Barrington Declaration.
The bottom line, from my perspective at least in terms of the Trump administration’s record with respect to federal policy related to SBM, is that President Trump was even worse than I had feared. That was even before his disastrous response to the COVID-19 pandemic and doesn’t even count his record on climate change and the environment. He sidelined scientists, prematurely approved treatments, and basically meddled in science-based policy in disastrous ways, leading to damage in the federal scientific infrastructure that could take decades to undo.
Will Joe Biden be better?
All of this leads to the natural question: Will Joe Biden, after he becomes President, be any better than President Trump on science? Certainly, it would be quite difficult for him to do worse, and there are several encouraging signs. For one thing, President-Elect Biden made controlling COVID-19 the centerpiece of his campaign, arguing that the economic damage due to public health mitigation measures can’t be reversed until the pandemic is under much better control. He has promised to listen to scientists and on Saturday announced that today he would be forming a COVID-19 task force full of real scientists to advise him during the transition:
“On Monday, I will name a group of leading scientists and experts as transition advisers to help take the Biden-Harris Covid plan and convert it into an action blueprint that will start on January 20, 2021,” Biden said. “That plan will be built on bedrock science. … I’ll spare no effort, none, or any commitment, to turn around this pandemic.”
The task force is effectively an extension of the group of advisers who have briefed Biden and Harris on Covid-19 for months. The group includes David Kessler, the former Food and Drug Administration commissioner; Vivek Murthy, the former surgeon general; and the Yale physician-researcher Marcella Nunez-Smith.
Biden has not yet named other task force members, but his campaign has received counsel from an array of well-known health policy figures in recent months, including Zeke Emanuel, the former Obama administration health adviser, and Nicole Lurie, the Obama administration’s assistant health secretary for preparedness and response.
Joshua Sharfstein, the former deputy FDA commissioner, is also seen as a contender to play a leading role in Biden’s Covid-19 response, and Ron Klain, the former Obama administration “Ebola czar,” is expected to play a major role in the Biden administration.
In a recent interview, Emanuel told STAT that the task force would likely feature “sub-task forces in each area” of pandemic response, including testing, contact tracing, and vaccines.
And:
He is also calling for “evidence-based national guidance” on when businesses like bars and gyms should open or close depending on the level of local transmission.
“I think the overriding theme here is that he understands the critical importance of a one government approach to address health emergencies,” said Howard Koh, a former assistant secretary for Health in the Obama administration who’s now at the Harvard T.H. Chan School of Public Health.
Another failure of the Trump administration was to refuse to use the resources of the federal government to provide guidance and instead basically leaving each state to fend for itself. Another disastrous decision was to pull the US out of the World Health Organization:
Biden’s administration will also re-open lines of communication with other countries and international organizations in its fight against the coronavirus. Trump pulled the United States out of the World Health Organization earlier this year, criticizing the international agency for supporting China, where the outbreak began. “Joe Biden and Kamala Harris understand that no country can face our current challenges alone and hopefully will re-engage and help re-form key science-based multilateral institutions,” says Marga Gual Soler, an expert in science diplomacy and policy adviser to the European Union.
Regular readers know that we haven’t exactly always been the biggest fans of the WHO, but our criticism has been in response to its naïvété about the antivaccine movement and its too-easy embrace of quackery in the form of traditional Chinese medicine and other traditional medicines, which it’s legitimized by incorporating them into ICD-11. However, when it comes to a global pandemic, not collaborating with WHO on responses to the pandemic was about as bad an idea as there could be.
Again, all of this is just about the pandemic. There are so many other areas of science where a President Biden is very likely to be so much better than President Trump was. Moreover, the horror of scientists over the response of this administration to the pandemic was not primarily due to politics, as Yale epidemiologist Gregg Gonsalves put it:
I'll just say this once. @realDonaldTrump would have had every scientist by his side if he had ever been serious about #COVID19. He turned to quacks, charlatans. Even a few hours after his defeat you can see what a real response might start look like without him.
— Gregg Gonsalves (@gregggonsalves) November 7, 2020
Precisely. It’s true that most scientists (although not most physicians) tend to lean left in their politics, but many are the scientists who have worked with Democrats and Republicans over the years to craft health and biomedical science policy. The horror of scientists, physicians, epidemiologists, and public health academics over the Trump administration’s response to the pandemic was far more due to the bungling and the hostility towards listening to scientists than it was to dislike of Trump’s politics. Yes, it is true that pretty much all administrations, be they Democrat or Republican, have a problem with “inconvenient science” that conflicts with political imperatives or messaging at some point or another, but never have I seen anything like the war on science carried out by the Trump administration.
This brings me back, at long last, to the reaction of scientists.
Why were scientists surprised in 2020 as well?
As I related, scientists were shocked at the outcome of the 2016 election. At the time, I wondered why, not so much because I myself wasn’t shocked that Trump was actually elected President, but more because so many of the responses were along the lines of, “How could we elect someone so hostile to science and science-based federal policy?” At the time, after some thought, my response was: Because to most voters, science and science-based policy are not the most important determinants of how people vote, often not even close. This was particularly true given the tsunami of misinformation and disinformation that social media served up in the run-up to the election.
So it was that I was rather disappointed to see the same sorts of responses to this election, specifically scientists expressing disappointment that the election was not an overwhelming repudiation of Donald Trump in that it was very close in several swing states to the point that the Electoral College was so close and that the Senate appears likely to remain in Republican hands. Indeed, on the day after the election, Nature published a news report entitled Scientists aghast as hopes for landslide Biden election victory vanish:
“It’s horrific,” says Kim Cobb, a climate scientist at the Georgia Institute of Technology in Atlanta. Cobb has her own fears about the pandemic, the economy and climate change but is trying to understand the anxieties that are driving so many to vote for Trump.
“It is depressing to see that the American electorate have not heeded the evidence of the last four years to give a strong message about the damage being caused by Trump’s actions and behaviour, for their own country as well as the wider world,” says Athene Donald, a physicist at the University of Cambridge, UK.
And:
Michael Lubell, a physicist at the City College of New York who tracks science policy, worries about what the results of the election say about the value that many Americans put on truth. Trump, former host of the television show The Apprentice, is well known for making inaccurate statements and spreading misinformation to further his political agenda. “What I see is people rejecting reality, and opting more for a reality show,” says Lubell. “And that to me is frightening.”
You might remember that Lubell was one of the scientists cited at the beginning of this post as having been horrified by Trump’s victory in 2016. As someone who has long advocated science-based health and medical policy, I understand his horror. What I don’t understand is his surprise, however. Again, science is not one of the major drivers of whom voters choose when they vote. Scientists seem not to understand this, which is somewhat understandable in that they are, well, scientists. However, this is not surprising in a nation this politically polarized, with such large segments of the population subjected to conspiracy theories about COVID-19, climate change, vaccines, and many other areas of science to the point where they believe antiscience conspiracy theories. Add to that the sheer number of people who believe in alternative medicine and don’t trust SBM, and I’m surprised that scientists would be so shocked at a close result. By contrast, the reason I was surprised by how close the result was (at least in the Electoral College) was primarily due to all the polling that showed Biden with such a stable and significant lead rather than because I expected the electorate to soundly reject him based on his antiscience attitude and policies. (I’m not bragging, either. I just understand this because I happen to spend so much time refuting pseudoscience and antiscience conspiracy theories.)
The article’s tag line is: “With so many votes cast for Trump in US election, some researchers conclude that they must work harder to communicate the importance of facts, science, and truth.” Personally, I can’t argue with that, but I also can’t help but note that there is still a disdain out there among my fellow scientists and physicians for scientists and physicians who choose to engage in science communication directed at the general public. It’s a disdain that dates back to Carl Sagan and before, an attitude that does not value such activity and even views it as a waste of time compared to other activities. While it is true that science communication is a lot more respected now (or at least a lot less disrespect and disdain are directed at it) by fellow scientists and physicians, we have a long way to go. Also, as much of a bubble as Trump supporters live in, we scientists have to be careful not to stay within our own bubbles so much. If we didn’t, we wouldn’t be surprised at occurrences like the results of the 2020 election.
Over the next several months, I think I’ll try to devote more verbiage to how we as medical scientists can try to promote good science, critical thinking, and reason in the face of the massive waves of disinformation that dominate social media. I’ll have to think about it, though. I hope those of you who share our mission will take the time to reflect as well on what we can do better. While it’s true that having Joe Biden as President will mean better science policy, the forces that elevated Donald Trump to the Presidency and supported his antiscience policies haven’t gone away and will make reversing the damage to federal science very difficult, even after the pandemic is finally brought under better control.

Acupuncture and Evidence Based Medicine
Steven Novella on July 31, 2024
Link

Over the last few decades there has been a steady infiltration of acupuncture into Western mainstream medicine. It is not unreasonable to conclude that acupuncture’s journey from an exotic Eastern practice to a fringe treatment to mainstream acceptance has been complete. Knowledgeable proponents of science based medicine (SBM) who remain skeptical of acupuncture now find themselves on the fringe. How did this happen?
Any complex cultural phenomenon is going to have multiple causes. Acupuncture is basically the poster child for all the inadequacies in academia and medicine in terms of how such issues are dealt with. Most medical professionals simply are uneducated about the details, they have out-sourced knowledge of acupuncture to the “experts” (proponents), there is a layer of reluctance to criticize born of cultural sensitivity, and there are still perverse incentives throughout both scientific and medical institutions.
But I will argue that core to acupuncture’s success is the rise of evidence-based medicine (EBM) starting in the late 1980s. Proponents of acupuncture have expertly exploited all of the weaknesses and shortcomings of EBM, which is why it is probably the best example of why we need SBM. As a treatment acupuncture is also optimally positioned to exploit EBM.
What Is Acupuncture?
We have explored acupuncture at length on SBM, but for a quick overview, acupuncture is the practice of sticking thin needles into specific points on the body in order to influence the flow of chi (a life force) to treat symptoms and diseases. Historically, acupuncture was more similar to blood letting than its fairly recent incarnation, and acupuncture points were essentially astrology. The whole using very thin needles and chi thing is a modern retconning of acupuncture created when various mutually incompatible traditions were united into “Traditional Chinese Medicine” (TCM).
A post-modern, if you will, further retconning of acupuncture by some contemporary proponents try to eliminate reference to chi or life force, and instead invoke a poorly defined and understood mysterious physiological mechanism from sticking needles into acupuncture points. But even in this formulation – acupuncture, by definition, requires sticking needles into acupuncture points.
EBM on Acupuncture
You can search for articles on SBM vs EBM here to get a detailed discussion of what we consider to be all the salient differences, and proponent of EBM will disagree on how best to implement EBM in practice (with some essentially morphing it into SBM and even claiming that is what it was all along). But at its core, EBM relies upon the highest quality clinical evidence to test the safety and efficacy of specific health interventions. While this sounds great, David Gorski has termed this “methodolatry” because it focusses too narrowly on this one method, while ignoring vast swathes of science.
A narrow focus on such clinical trials allows for a lot of abuses – including p-hacking, publication bias, poorly defining terms, inadequate blinding, and mixing variables. SBM takes a more thorough approach that tries more rigorously to control for all these things by looking at the overall patterns of evidence in the literature. SBM also explicitly considers scientific plausibility and prior probability, and prefers a more Bayesian approach to statistics.
In brief – acupuncture exploits all of these weakness of EBM. Acupuncture studies mix in variables like “electroacupuncture”, allow study designers to choose acupuncture points ad hoc without any internal consistency, rely heavily on poorly blinded or even unblinded comparisons, and claim that when “placebo” or “sham” acupuncture work as well as “verum” or true acupuncture, that just means that these other forms of acupuncture also work. P-hacking is rife within the acupuncture literature. Further, when looking at studies from China, publication bias is literally 100%.
Acupuncture also is almost designed to maximize placebo effects. It is an elaborate ritual, involves an invasive procedure, and involves a lot of hands-on attention and positive encouragement from a practitioner.
But – using EBM you can generate some statistical outcomes that proponents can point to in order to claim that acupuncture is “evidence-based”. This has become a powerful tool for promoting acupuncture to mainstream professionals. There is also just enough superficial plausibility, since something physical is actually happening (unlike, say, in homeopathy). Acupuncture researchers have also focused on the subjective symptoms that are most amenable to placebo effects to make their case.
The SBM Approach
When stepping back and taking a full SBM approach to acupuncture, a very different picture emerges than the narrow EBM approach. The plausibility of acupuncture is very low, because there is no known mechanism, and outside of short term pain and nausea reduction, for any medical application of acupuncture there is no plausible even theoretical mechanism.
Perhaps most devastating is that, even after extensive research, acupuncturists cannot demonstrate that acupuncture points exist. They cannot even agree on where they are and what they do. In short – scientifically speaking, acupuncture points don’t exist. If that’s the case, than acupuncture itself does not exist. From a basic science perspective, acupuncture fails utterly.
From a clinical research perspective, it also fails. Systematic reviews have failed to show a consistent replicable and specific effect from acupuncture. This is true despite decades of research and thousands of studies. If there were a real effect there, we should be clearly seeing it by now, but we are not.
What the clinical research does reflect is mostly small and poorly designed studies, with the better studies tending to be negative. There is lots of p-hacking and publication bias. There is also a great deal of heterogeneity – different acupuncture points and different outcomes. In other words – the acupuncture literature in total has all of the red flags and features of an intervention that does not work. If you compare this with a dubious history and lack of plausibility, the only SBM conclusion is that acupuncture does not work.
And yet it is increasingly promoted as an evidence-based treatment. This, however, reflects only the failure of EBM to properly address the question. 

Is There a Role for the Art of Medicine in Science-Based Practice?
Clay Jones on June 20, 2014
Link
The practice of medicine is an art, not a trade; a calling, not a business; a calling in which your heart will be exercised equally with your head.
The practice of medicine is an art, based on science.
-Sir William Osler, AEQUANIMITAS
The truth is that many of us have some kind of “extraordinary gift.” For a few of us, that gift is the ability to throw a ball at 90 miles per hour and hit a catcher’s glove. For others, that gift is a form of extraordinary perception. Medical intuitives “see” things that others don’t. Wendy Marks has been described as a “human CT scan.” What no one has been able to diagnose by conventional methods is often seen when Wendy scans a body.
–Boston Women’s Journal April/May 2002
The concept of an art to the practice of medicine comes up frequently and in a variety of contexts. Early on in our medical education, we are exposed to the phrase and what it supposedly means, which I will discuss in more detail shortly. But the art of medicine is always painted (pun intended) in a positive light. I will admit that I have a strong opinion, perhaps biased by my involvement with the science-based medicine movement and an equally early exposure during my medical training to champions of evidence-based practice and the use of critical thinking in the approach to patient care.
But first, imagine the following four scenarios:
     1.  A patient seen by an emergency medicine physician for complaints of chronic mild and intermittent periumbilical abdominal pain has a normal physical exam. The physician establishes rapport with the patient while efficiently discussing her past medical, surgical, social and family history among other components of a thorough investigation. The patient is reassured that her normal exam, and a history absent any concerning red flags for organic illness, do not warrant additional laboratory or radiologic studies at this time. The physician discusses the potential role of stress in chronic abdominal pain and recommends that the need for outpatient mental health services be discussed with her primary care physician as well as implementation of lifestyle modifications such as increased exercise and a healthy diet with adequate intake of fiber. The patient is educated on possible concerning new signs or symptoms that would necessitate further work up.
     2.  The same patient is seen by her primary care physician the following week. She has a normal physical exam. The physician establishes rapport with the patient while efficiently discussing her past medical, surgical, social and family history among other components of a thorough investigation. The patient is told that while her normal exam, and a history absent any concerning red flags for organic illness, are reassuring, in her experience some serious conditions can present without obvious findings in the early stages. The physician relates the story of a similar encounter during residency to one of her colleagues and sends the patient down the street for a number of metabolic studies and an abdominal CT to rule out inflammatory bowel disease. She is referred to a gastroenterologist.
     3.  The patient is then seen by a local gastroenterologist one week later. She has a normal physical exam and no concerning red flags are discovered during brief questioning. The physician is in the room for about ten minutes when he abruptly recommends endoscopy to “see what’s going on in there” and shows her to the clerk for scheduling of the procedure. She is given a packet of information on the risks and benefits of the procedure and a prescription for medicine to clean out her bowels in preparation. She is reassured that biopsy findings will likely be normal and told that it is best to figure these things out so that she can quit worrying.
     4.  The patient, having failed to show up for the endoscopy, is seen by a practitioner of Naturopathy recommended by a friend. The practitioner establishes rapport with the patient while discussing her past medical, surgical, social and family history along with a number of questions about potential exposures to chemicals and toxic substances in her environment and diet. The patient’s concerns are validated and she is warned that often problems like systemic yeast and various nutritional imbalances can present with vague symptoms but may become more debilitating. The practitioner orders a variety of metabolic studies as well as tests for heavy metals in the patient’s saliva, and he performs live blood cell analysis in the office. A number of supplements and a homeopathic remedy are prescribed, and acupuncture sessions are scheduled.
At first glance, especially to those who are familiar with and support the ideals of science-based medicine, it’s easy to see which of these scenarios is full of blatant pseudoscience and a good example of what goes on in the world of alternative medicine. Sadly, the practitioner of Naturopathy probably convinced our patient that she suffers from one of the many “one true causes” of all disease. You also probably realize, even without any specific medical training, that the physician in scenario 1 provided the most science-based approach to the patient’s symptoms, with excellent communication skills and an admirable level of comfort with a likely stress induced complaint.
In scenario 2, a well-meaning and competent personal physician perhaps placed too much importance on personal experience than was deserved, and I imagine this fictional doctor probably eschews what she perceives as excessive reliance on cookie cutter algorithms in “the real world.” The subspecialist in scenario 3 certainly didn’t demonstrate a good bedside manner, but probably meant well in his attempt to put his patient’s mind at ease with what is arguably an unnecessary invasive diagnostic procedure. All of these practitioners would almost certainly claim to be practicing the art of medicine, however, which should raise the question of just what the heck it means to do so.
What is the art of medicine?
This is a question I have often asked myself. When doing so I am frequently reminded of how politicians present themselves to the public in radio and television spots, which essentially amount to “Candidate Jenkins is for good things and against bad things!”, or for the bad and against the good if you happen to fall on the other side of the political spectrum. But they don’t really tell you anything about the candidate as a person. In a similar fashion, talk of the art of medicine is almost always focused on the positive aspects. Always a Rembrandt, never a Bush.
According to Sharon Bahrych, a Denver-based physician assistant, in her post on KevinMD, the art of medicine involves several components:
     1.  Caring for patients, showing honest concern and compassion
     2.  Giving patients time, not rushing in and out of the exam clinic room, being patient with them, having a great bedside manner
     3.  Using the evidence based medicine algorithms as a guideline, as we apply them to each and every patient we see. Understanding that every patient is an individual who has individual circumstances that affect their lives.
     4.  Helping every patient to acquire the best outcome they can for themselves by working with them, educating them, coming up with a mutually agreed upon plan of action
This is in keeping with most examples of how the art of medicine is defined. It’s about compassion, communication, professionalism, respecting patient autonomy, treating each individual as a beautiful and unique snowflake, and not being afraid to go off protocol when things get squirrelly. It all sounds very nice and hard to argue with.
It’s particularly hard to argue with the narrative when people start dropping quotes from Sir William Osler, like I did at the onset of the post. Osler, widely considered to be the father of modern medicine, accomplished many things during his career, not the least of which was the medical residency as an entity and the push to teach the practice of medicine at the bedside more so than in the classroom. An even-brief skimming of his biography should be sufficient to leave any physician feeling rather inadequate.
Osler’s legacy consisted of much more than this. A variety of medical conditions and specific findings bear his name, as do many buildings. He also left us with a seemingly endless supply of quotes, with several appropriate candidates to choose from for the title slide of almost any PowerPoint presentation. His notable quotes regarding the art of medicine speak volumes. Osler called for physicians to care for the patient, not the disease. To employ empathy, humanity and love, and accept that uncertainty and error are unavoidable in the practice of medicine. And to respect experience as a means of learning the art of medicine, but never forget that the art is subservient to the science.
The most compelling aspect of the art of medicine involves communication. While it really doesn’t matter all that much if you are an excellent communicator full of empathy and love for a patient who was diagnosed with an ear infection, the next patient may not be so simple. As Harriet Hall once wrote, “Medicine is not an art like painting. Neither is it a science like physics. It’s an applied science.” While I often, with tongue in cheek, make the claim that a robot could perform a lot of my job successfully, nobody wants to be told that they have cancer by a machine. The application of science to the patient will always need a human component.
Is there a dark side of the art of medicine?
If, like the Force in the Star Wars Universe, the art of medicine manifests itself as our “compassion, selflessness, self-knowledge and enlightenment, healing, mercy and benevolence”, then it seems fitting that there would also be a dark side. As water takes the shape of the vessel which contains it, the fluid nature of this aspect of medical practice allows misuse in the wrong hands. There is, after all, an “art” to more than just medical practice. Proponents of chiropractic, acupuncture and naturopathy, not to mention every other alternative medicine modality, incorporate similar language into their propaganda.
The dark art, so to speak, of medical practice is often employed as a rationalization when ignoring established evidence. Medicine can be very complicated, messy even, but a great deal of it is actually fairly straightforward. We have picked a lot of the low- and even high-hanging fruit in regards to both prevention and treatment of injury and illness. And when we can’t cure, we can often manage symptoms and improve the quality of a patient’s life. Of course there are gaps in our knowledge, but those gaps are steadily shrinking. The so-called art of medicine does often thrive in those gaps, and that can be a good thing, but the gaps are also where there is the greatest potential for harm when seen as a license to do whatever you want regardless of plausibility and basic science.
What many people think of as an art in medicine is the ability to make a diagnosis. Mark Crislip once referred to it as a craft rather than an art, but I just call it pattern recognition. Dr. Crislip went on to further elucidate his concept of the art of medicine:
The Art in medicine may occur as thinking about cases moves more and more over time from the conscious level of a third year medical student to the subconscious level of an experienced clinician. I recognize subtleties and important findings faster than newbies or the inexperienced.
Over the decades, as evidence has accumulated, we have learned a vast array of patterns and associations that are beyond the ability of most if not all individual physicians to maintain a complete working knowledge of. Hence we must increasingly rely on specialists who focus on specific areas of medicine and, when possible, the incorporation of the available evidence into algorithms that fit most patients most of the time. It is hard to deny the improved accuracy and outcomes that this approach has led to.
Certainly there will always be exceptions where patients don’t present in such a way that an applicable protocol is easily chosen or even available, and diagnosis does elude us at times. And patients will not always respond to treatment as we expect them to. But we must be wary of experience-based practice that focuses too much on individualization. Without due caution, and awareness of the many biases and errors in perception that plague us all, the difference between expert-level subconscious pattern recognition and the bogus concept of medical intuition may not always be clear.
In the world of alternative medicine, the complexity of medicine is exchanged for the simplicity of pure invention and excessive generalization (everybody has a spinal subluxation), even while attacking medicine for not being holistic enough. While we may quibble about how much of a role art plays in medicine, it is all that alternative practitioners have to offer really, with rare exception. And examples of our occasional difficulty applying what we know to the individual patient commonly serve as justification to ignore scientific progress entirely.
Physicians aren’t immune to this. The art of medicine is often described as how we apply the science to the individual, but taken too far it can serve as a shield against criticism, a place for quacks of all kinds to take shelter against evidence. The art of medicine is all-too-often used as a conversation-ending justification for what is simply bad medical practice.
Conclusion
In my opinion, the phrase “art of medicine” needs to be retired or at least restricted in use. Aside from being nebulous to the point of nearly losing all meaning, and serving as fertile ground for all manner of bogus approaches to healthcare, to me it almost implies that there should be some kind of innate ability to practice medicine that some people have and some don’t. I don’t believe that there are medical equivalents of Leonardo da Vinci or Yo Yo Ma, virtuosos with skill that can’t be acquired by practice alone.
But we shouldn’t throw the baby out with the bathwater. While I certainly understand the desire to have an all-encompassing ideal phrase, I believe that we would be better served to be more specific when talking about the important aspects of being an effective healthcare professional, and more honest about when we are making an educated guess secondary to a lack of evidence. When a legitimate healthcare professional chalks their action up to the art of medicine, it lends credence to when a quack does the same thing.
Good communication, in my opinion, stands out as the most legitimate aspect of the art of medicine. While it is true that some people do have a knack for it, and even that some physicians will probably never develop into excellent communicators, the vast majority of physicians can be taught how to speak to patients competently. And preternatural communication skills aren’t called for very often in medicine.
I don’t know if the net positive benefit of focusing on the art of medicine is worth the negative. It probably is. Think of this as a conversation starter and please let me know your thoughts in the comment section. I feel very comfortable saying though that we would certainly be a lot better off if we focused more on critical thinking during medical training. A strong foundation in scientific skepticism would very likely decrease the potential for abuse of the art of medicine.

On humility, confidence, and science-based surgery
David Gorski on June 24, 2013
Link
Every so often, the reality of trying to maintain a career in science-based medicine interferes with the fun that is writing for this blog. Basically, what happened is that I spent the entire weekend working on three different grant applications and, by the time Sunday night rolled around, I was too exhausted to write what I had originally planned on writing. Fortunately, one advantage of having been blogging so long and also having blogged under a pseudonym over at my not-so-super-secret other blog is that there’s a lot of material which is pretty damned good, if I do say so myself, that I can draw on for just these situations. Even better, it’s old enough that it’s unlikely that most of you have actually come across it before, which makes it new to you (well, at least most of you). As a special bonus, the jumping off point was a post by an occasional contributor to this blog, Peter Lipson. Actually, I wish Peter would contribute more regularly, but he’s too busy moving on to bigger things at Forbes.
This time around, I’m half-recycling, half-revising a post that was a bit more navel-gazing than usual. However, as the only surgeon on SBM I think it’s actually useful every now and then to discuss the trials and tribulations of practicing science-based surgery. It began when Peter wrote an excellent meditation on a topic that’s always been a difficult issue for me to face as a surgeon, namely how one balances confidence in one’s ability with humility in the face of disease and uncertain science. He started with a spot-on observation:
The practice of medicine requires a careful mix of humility and confidence. Finding this balance is very tricky, as humility can become halting indecision and confidence can become reckless arrogance. Teaching these traits is a combination of drawing out a young doctor’s natural strengths, tamping down their weaknesses, and tossing in some didactic knowledge.
Peter then went on to describe how he tries to teach physicians-in-training the right balance and does an excellent job of it. He’s absolutely right about pushing residents to make a decision and justify it to the attending, trying to get them to think like an attending who knows that the buck stops with him or her while they are still in the safe confines of the training program, with a real attending covering their backs.
In surgery, I think, the mix is different. Surgeons have to project confidence to their patients because the technical skill involved makes surgery and other procedure-driven specialties somewhat different. It’s not just about knowing the science. It’s about being good at the technical skills that are so important in surgery. As much as it might be denied, proposing a course of treatment that involves cutting into a person and rearranging his or her anatomy for therapeutic effect is perceived differently by patients than proposing they take a new medication or even undergoing chemotherapy. It’s far more invasive and far more dependent on the raw skill of the practitioner. Teaching surgery is also different than teaching internal medicine because there are two elements involved. Like the case for medical specialties, there is the cognitive element, teaching diagnostic skills and the appropriate therapies for various diseases and conditions. However, in addition to these cognitive skills that must be taught, there is also a huge base of technical skills that must be mastered. It’s true that internists and other physicians must also learn a number of technical skills, such as placing IVs and central lines, doing lumbar punctures or bone marrow biopsies, and tapping pleural effusions, but, even in the more procedure-oriented internal medicine specialties, the number and level of invasiveness of these procedures are nowhere near what surgeons must learn. In brief, no therapy can mess a patient up if it goes wrong quite like surgery, and every surgeon who does large cases has at some point in his career messed up a patient. That doesn’t make them bad surgeons; it’s the nature of the beast. No matter how good a surgeon is, complications are inevitable. How many complications and how the surgeon deals with them are what separate good from bad surgeons.
This realization makes the proper balance between confidence and arrogance arguably more difficult to reach. We surgeons have all encountered at some time in our careers the “cowboy” surgeon. This variety of surgeon seemingly has no fear and will plunge into even the most difficult and dangerous cases (“The patient has diabetes, severe coronary artery disease with an ejection fraction of 25%, COPD, and is on aspirin and Plavix? No problem! Let’s operate!”), seemingly oblivious to the risk. He not infrequently gets into trouble (“Oops, I severed the aorta! Don’t worry. I can fix this.”) but usually manages to get out of it, seemingly unfazed by the experience and the close call that the patient had. In contrast, we have also all encountered the excessively cautious surgeon, the one who often hesitates and seems almost afraid to operate, even when the situation calls for, as I like to put it, maximal invasiveness. Both are extremes that a surgeon should try to avoid.
Add to this mix patient expectations. When I first started practicing after leaving my fellowship, I thought that the best approach was to lay out the surgical options, the risks and benefits of each based on my best interpretation, and to try to let the patient decide, with my advice as needed. I soon found that this was a problem. Reports came back to me through my division chief that some patients viewed me as indecisive and didn’t have the confidence in me necessary to let me operate on them. Then I learned that this wasn’t the case for all patients. Some genuinely liked this approach because to them it respected as much as possible their autonomy. Others hated this approach because they had expectations of what a surgeon should be, and those expectations included telling them what needed to be done and just doing it. No doubt the same is true of patients in other specialties, but the sheer invasive and personal nature of surgery tends to shift the balance of patient expectations more towards the paternalistic model. Surgeons see things in a patient that even their spouses never see, namely their insides, and this, coupled with the knowledge that it is the skill of one individual that can determine success or failure of even the correct course of action, makes surgery very intimate and personal to the patient.
What I eventually learned was that not only does a surgeon have to find the right mix between paternalism and doing what the patient wants, between confidence and arrogance, all the while choosing courses of action that are supported by science, the surgeon must also be able to size up patients to figure out what specific balance between these competing traits each individual patient expects and then titrate his behavior accordingly. Some patients really do just want the surgeon to tell them what needs to be done and then to do it, without all that confusing discussion of options based on the surgical and scientific literature. Such patients frequently ask the question, “What would you recommend if I were your wife/mother/sister?” The surgeon had better be able to give the answer to that question with confidence and still tell the patient enough about the risks to obtain truly informed consent. Others want a full discussion to the point of wanting references from the peer-reviewed scientific literature, in which case the surgeon has to titrate his demeanor to a less paternalistic manner. I like to think I’ve gotten better at this in the last 15 years. Certainly I haven’t heard word of patients viewing me as indecisive in a while. (Cue a patient finding this blog and telling me how wishy-washy I am.)
In terms of training, the way Peter described training young physicians is certainly operative–if you’ll excuse the term–in training surgeons in the nonsurgical skills of diagnosis and nonoperative treatments of surgical diseases. Indeed, the best teachers I ever had did exactly that. Many are the times I recall calling an attending in the middle of the night and, after telling him about the patient, hearing the response, “OK, what do you want to do now?” Woe be unto me if I didn’t have a well-reasoned plan of action. Indeed, it was better to have a bad plan of action than to stammer back, “I don’t know.”
The differences between surgical and medical training become most apparent in the operating room. The art of teaching a young surgeon how to operate is incredibly difficult. Indeed, when I was a resident, I never appreciated just how difficult it is for a surgeon to take a resident through a case and keep his or her sanity. Now that I’m on the other side of the operating table, I know. When the resident falters, there is a very strong tendency to want to grab the instruments and take over the case, but doing so too quickly will prevent the resident from learning how to do difficult dissections or to handle other difficulties encountered in the OR. On the other hand, patient safety must be paramount. Letting the resident struggle too long (for instance, trying to dissect a structure free from a large blood vessel) runs the very real risk of harming the patient, and that can never be allowed. I remember well one attending that I had whose wisdom I didn’t appreciate at the time. He leaned more towards the “cowboy” type of surgeon but his skills were so legendary that he really could almost always get himself out of any trouble that he found himself in. He forced residents beyond what they thought they could do, although he frequently yelled as they did it. What I realized later is that he was just so technically gifted that it drove him crazy to watch me and other residents clumsily try to do what he could do with slickness and utter aplomb, but he restrained himself from taking over the case unless the patient was in danger because he was just that dedicated to teaching. He also taught me a number of things that no other attending did, such as how to take down bowel adhesions with the knife instead of bluntly or with scissors, how to do a Stoppa hernia repair, and a number of other maneuvers that I still use to this day.
Another aspect of surgery that makes it difficult to avoid arrogance is that surgeons tend to have a mentality that surgery can fix things. And fix things it most definitely can, sometimes in a truly dramatic and satisfying fashion. The problem is, however, that because it is so difficult (and often impossible) to do truly “gold standard” randomized, double-blind studies on surgical therapies, the level of evidence supporting them is often based on a preponderance of retrospective studies and other inferences. This makes surgery, at least in my experience, more prone to the persistence of dogma beyond when scientific and clinical evidence doesn’t support a therapy anymore. Surgery residencies also tend to function in a much more hierarchical manner; indeed, I have often likened them to the military, with a clear chain of command that is violated at one’s peril. Interns usually don’t go straight to the attending without going through the chief resident first, and orders tend to flow downhill from the attending, to the chief or senior resident, all the way down to the junior residents. Medicine residencies tend to be different, with less of an emphasis on rank. It’s not that rank doesn’t matter; it’s just that it doesn’t seem as rigid as in surgery residencies. Although it is changing, probably in response to overall societal changes that are less tolerant of rigid authority structures and mandated work hour limitations for residents, which increasingly force attendings to deal with whatever resident is there (often an intern), vestiges of a military-like hierarchy still remain and likely will remain. This can lead to what I call “tradition-based” surgery, typified by the remark, “This is how I was taught to do it and how I’ve always done it.”
The flip side of this ability to “fix” things is that surgeons really do love bright, shiny, new surgical procedures and technology. In other words, surgeons (as a specialty) have a distressing tendency to be susceptible to “bandwagon” effects. I’ve written about this before with respect to the rapid adoption of laparoscopic cholecystectomy years before clinical data demonstrated it to produce equivalent relief of symptoms with an acceptable complication rate. “On the ground,” laparoscopic cholecystectomy looked so dramatic in its ability to alleviate symptoms of gallbladder disease with a greatly decreased level of pain and time to recovery from surgery, but until the clinical studies were done it was impossible to know if the long term complication rate, particularly the rate of bile duct injuries, was unacceptably high. To balance this out, however, I’d be remiss if I didn’t mention once again that some of the best and most rigorous controlled studies (such as in breast cancer surgery) were done by surgeons. These two tendencies are often in conflict in surgery and must be balanced, and it’s not easy.
Finally, Peter mentioned that excessive arrogance in the face of disease and science can lead to quackery and crankery:
It takes years of training to develop the decision-making skills that go into being an effective attending physician.
This is one place where we part ways with the cranks and quacks.
Cranks and quacks lack humility in the face of disease. They have confidence without knowledge. As a real doctor, I know, with complete certainty, that I will have failures. I know that there are some diseases I can’t beat. The variety, complexity, and horror of human diseases have taught me my place. I can’t promise miracles, but I can give statistics.
Quacks and cranks do promise the improbable. They promise to stop you from aging. They promise to stop autism by fighting vaccination. They promise to twist your chi until your malaise relents. Most important, they don’t know what they don’t know, and that makes them dangerous.
I sometimes wonder if surgical training and surgical culture, with its emphasis on confidence and action over introspection, makes surgeons particularly prone to quackery and crankery. Certainly, Dr. Roy Kerry, the head and neck surgeon-turned quack whose quackery killed an autistic child gives me pause, as does the case of Dr. Lorraine Day, a prominent academic orthopedic surgeon who embraced not only quackery but all manner of conspiracy theories, including Holocaust denial. Another thing that gives me pause is the number of surgeons who seem to embrace “intelligent design” creationism, including a prominent neurosurgeon and a general surgeon. I realize it’s a small sample, but I tend to wonder whether surgeons seem especially prone to the arrogance of ignorance when it comes to areas outside their expertise and prone to their confidence leading them astray within their field.
The bottom line is that practicing evidence- and science-based surgery is, as for all specialties, exceedingly difficult. Balancing the confidence to make a decision and persuade the patient of its correctness with humility in the face of disease, uncertainty, and conflicting evidence, leavened with a firm knowledge of facts and the scientific method sufficient to allow a surgeon to interpret the data in the light of his or her own experience and the unique situation of the patient and then apply that interpretation in a manner most likely to benefit that patient, represents the core of surgical excellence. This knowledge and these skills are very hard to acquire and teach, but not by any means impossible if the teacher wants to teach them and the learner wants to learn them.

Two Viewpoints
Mark Crislip onJ une 14, 2013
Link
Most of what I read professionally is directed towards reality-based medicine. I spend my professional energies thinking about the application of reality to killing various and sundry microscopic pathogens.
The conceptual framework I use, and that used by others in medicine, does not concern itself with the application of the Supplements, Complementary and Alternative Medicines that occupy the attention of this blog. In acute care medicine SCAMs are of virtually no importance yet the approaches we need to take with patients and medicine are, with slight changes in emphasis, as applicable to SCAMs as real medicine. You need to remember, however, that the topic is not necessarily based in known reality.
Two viewpoints in JAMA caught my attention this month, both more thoughtful and reasoned than I am probably capable of. While focused on the application of reality-based medical practice, they apply to the topics of SBM as well.
The first is Evidence-based persuasion: an ethical imperative.
Evidence-based persuasion. At some level the raison d’être of this blog and the antithesis of the SCAM world. That it is considered an ethical imperative makes its lack of use in the SCAM world all the more damning.
The article points out in the introduction that:
There are at least 3 different types of persuasion. The first is the removal of bias. The second is recommending a particular course of action and providing evidence and reasons in favor of it; and the third is the potential creation of new bias, which could cross the line into unethical manipulation.
They go on to give examples applied to the practice of medicine. How about science-based medicine?
The first kind of persuasion, the removal bias, is the primary theme of this blog. Readers and writers of this blog are aware of all the types of bias that can warp judgment. I have long said that the three most dangerous words in medicine are “in my experience” because experience is unreliable in helping decide what works. Experience in medicine is the worst bias.
Ignoring experience is an unnatural way for humans to behave. Everything we do is a result of experience. The best restaurant in town*? The fastest way to work? Best headphones? In every aspect of life we rely on our experience and that of our social network to decide what to do. And then we get to medicine, the attempt to heal illness and relieve suffering, and we are asked to lay aside a lifetime’s approach to the world and rely on clinical trials? Not likely.
I had a patient just a few weeks ago ask me if she could take colloidal silver for her infection and I told her it does nothing. She countered that I was wrong, that she had used it many times in the past and it had always cured what ailed her. I knew I had not a chance in hell if convincing her otherwise for as Groucho said, who are you going be believe, me or your lying eyes?
Perhaps people are able to alter their biases when presented with the evidence, but I am not sure everyone is capable. When someone suggests that the reason I recommend vaccines, or any other reality-based therapy, is because I am a paid shill of big Pharma, I know that we inhabit two radically different realities that do not overlap. Such sentiments are not uncommon:
…one in seven Americans think the pharmaceutical industry is colluding to “invent” new diseases in order to profit off them…
Weird. Sure Big Pharma, like all companies, can behave with all the ethics of a psychotic shark, but the conspiratorial nature of some biases is just nuts.
In medicine when we discuss diagnostic and therapeutic interventions we sometimes have to dissuade people of erroneous ideas that could prevent them from accepting care. It is rare in my world. Most people, when acutely infected, accept the interventions I have to offer since the alternatives are rather unpleasant. The only common interaction is the occasional new AIDS patient who refuses HAART because they are convinced the medications are toxic and kill people. After explaining the history of HIV treatment I usually convince them to give it a try. As a result I have many patients who would have died in months in the bad old days who are now alive a decade later. Very satisfying.
But what if your whole practice is based on bias, on unreality, and you cannot realize it? The only bias you can alter is to convince your patients that real medicine is fantasy and that fantasy is reality. Welcome to Natural News, the bizarro world of medicine.
How about the second? “Recommending a particular course of action and providing evidence and reasons in favor of it.” Hard for a SCAM provider.
In my practice, hospital-based acute infectious diseases, it is reasonably simple. I know most of the pertinent literature for the common infections and if I have some weird bug in an odd place I research the problem and tell the patient the whys and wherefores of the proposed treatment. I know the science, I know literature (not always the same thing) and I know the best options.
What about a homeopath or acupuncturist or reiki practitioner? Can an Integrative Medicine Department ethically offer using these therapies after comparing them to the known world?
It is an interesting psychology: based on nonsense that is a polar opposite to the understanding of reality, the only favorable evidence that can be offered is “in my experience.’ It is a curiosity that real medicine uses what can be the least convincing arguments, those from the literature, while the homeopath has to rely of the least valid but most powerful argument, experience.
My patients often want to know what kind of experience I have. Has this worked before, how many of similar cases have I managed, and what would I suggest if it were my mother? I am always slightly unnerved with the question because I know how faulty my memory is, especially after almost 30 years of medicine. That’s maybe 25,000 cases. Like I can remember? But that is all the average SCAM provider has to offer.
The last form of persuasion, that of creating new biases, is the most interesting. It is an interesting balance. Patient autonomy is paramount in US medicine. They are the captain of their ship and it is my job to give them my best opinion as to their diagnosis and treatment. On the other hand, the process of explanation will persuade them and we all know the context of how information is given can create bias.
However all SCAM is about creating new biases that are divorced from reality.
It would interesting to get an ethics consult and ask the question of a hospital’s integrative medicine department if they can live up to the recommendations of ethical persuasion:
1) Remove bias and access the patient’s autonomous wishes2) Provide honest, impartial, evidence-based information about prospective harms and benefits3) Provide a rational interpretation of this information including facts about the belief set and views regarding the best decision4) Use reason rather than emotion while sometimes appealing to the patient’s emotions to counterbalance their existing emotional responses5) Avoid creating new biases6) Be sensitive to the patient’s changing preferences because persuasion is likely to change the patient’s outlook and perspectives.
The heart of all SCAM is in violation of the first 5. Given they are not based on known reality, they cannot follow those recommendations and it should be unethical to offered in a real medical environments.
However, SCAMs make money, and where money is concerned, rationalizations will follow.
The other viewpoint that caught my eye in JAMA was Synthesizing evidence: shifting the focus from individual studies to the body of evidence.
The first sentence is intriguing:
The research enterprise behaves as if a single study could provide the ultimate answer to a clinical question.
The rest of the essay is an argument that more emphasis should be placed on the results of meta-analyses and not rely on single studies.
I do and do not agree with the authors, but there are always caveats.
One of the issues that has always annoyed me with meta-analyses is that often as new studies are done they are incorporated into the prior analysis but the older, often more poorly done studies are not thrown out, so the bad studies tend to pull down the good ones.
The Cochran Collaboration has a nice overview of the systamatic review process but they are done under the implicit assumption that what they are reviewing are studies that evaluate reality. The Cochrane Reviews usually fail when they apply their methods to topics such as homeopathy or acupuncture, where positive results are always due to bias.
Even though they assess the quality of the studies, they do not take into consideration the prior plausibility that renders most SCAM studies suspect.
There is an arc in the literature concerning most SCAMs. Better and better quality studies demonstrate less and less efficacy until well designed studies demonstrate no effect.
The potential for that arc, as best as I can tell, is not part of the systematic review, but would give a hint as to the validity of studies where the intervention is divorced from reality. A plot of study quality vrs efficacy over time.
Part of my job is that of data synthesizer. What is the best way to treat, say, MRSA pneumonia? It can depend on many factors, some of which are not clear cut or have no data at all. Is the flu vaccine of benefit? The answer depends on what is considered a benefit and in what population; a single meta-analysis that looks at PCR-proven influenza will not include the effects on pregnancy, heart attack and stroke or the lack of spread to populations not vaccinated.
Whether or not a single study is superior or inferior to the collected wisdom of a systematic review depends on the question being asked and the plausibility of the intervention being studied.
I am not so certain that systematic reviews on fiction are a reliable way to understand the therapeutic efficacy of that fiction, but outside that caveat I agree with the authors conclusions: “It is time to focus on the entire body of evidence.”
And the body of evidence concerning most SCAM, as this blog demonstrates repeatedly, is there is no there there.
*Castagna.

“Moneyball,” the 2012 election, and science- and evidence-based medicine
David Gorski on November 12, 2012
Link
Regular readers of my other blog probably know that I’m into more than just science, skepticism, and promoting science-based medicine (SBM). I’m also into science fiction, computers, and baseball, not to mention politics (at least more than average). That’s why our recent election, coming as it did hot on the heels of the World Series in which my beloved Detroit Tigers utterly choked got me to thinking. Actually, it was more than just that. It was also an article that appeared a couple of weeks before the election in the New England Journal of Medicine entitled Moneyball and Medicine, by Christopher J. Phillips, PhD, Jeremy A. Greene, MD, PhD, and Scott H. Podolsky, MD. In it, they compare what they call “evidence-based” baseball to “evidence-based medicine,” something that is not as far-fetched as one might think.
“Moneyball,” as baseball fans know, refers to a book by Michael Lewis entitled Moneyball: The Art of Winning an Unfair Game. Published in 2003, Moneyball is the story of the Oakland Athletics and their manager Billy Beane and how the A’s managed to field a competitive team even though the organization was—shall we say?—”revenue challenged” compared to big market teams like the New York Yankees. The central premise of the book was that that the collective wisdom of baseball leaders, such as managers, coaches, scouts, owners, and general managers, was flawed and too subjective. Using rigorous statistical analysis, the A’s front office determined various metrics that were better predictors of offensive success than previously used indicators. For example, conventional wisdom at the time valued stolen bases, runs batted in, and batting average, but the A’s determined that on-base percentage and slugging percentage were better predictors, and cheaper to obtain on the free market, to boot. As a result, the 2002 Athletics, with a payroll of $41 million (the third lowest in baseball), were able to compete in the market against teams like the Yankees, which had a payroll of $125 million. The book also discussed the A’s farm system and how it determined which players were more likely to develop into solid major league players, as well as the history of sabermetric analysis, a term coined by one of its pioneers Bill James after SABR, the Society for American Baseball Research. Sabermetrics is basically concerned with determining the value of a player or team in current or past seasons and with predicting the value of a player or team in the future.
There are a lot of parallels between moneyball and “evidence-based medicine” (EBM), as you might imagine, as Phillips et al point out:
In both medicine and baseball, advocates of evidence-based approaches argued for the enhanced vision of statistical techniques, which revealed what tradition or habit had obscured. The difference between an all-star and an average hitter, for example, works out to about one hit every other week, a distinction that’s almost impossible for even a trained scout to recognize. Statistical power can be as relevant as opposite-field hitting power in the assessment of players. Early proponents of controlled medical trials similarly pointed to how difficult it was for an individual practitioner to determine a treatment’s efficacy or distinguish real effects from apparent ones after seeing only a small number of clinical cases. Mathematical measurements and calculations were meant to push practitioners away from naive visual biases — a player who “looks right” or a therapy that seems to work. Walks are far more important than they first appear in baseball; walking is more important than it first appears in medicine.
Moneyball has also entered politics in a big way over the election cycles of 2008, 2010, 2012. In the run-up to the 2012 election, I, like many others, became hooked on FiveThirtyEight, a blog devoted to applying rigorous statistical analysis to the polls. (FiveThirtyEight refers to the number of votes in the Electoral College.) As political junkies (and even many casual observers) know, the man responsible for the blog, Nate Silver, got his start as a “moneyball”-style sabermetrics baseball analyst. In 2002, he developed a model to assess and predict a baseball player’s performance over time, known as PECOTA, which stands for “Players Empirical Comparison and Optimization Test Algorithm.” Silver brought his model to the Baseball Prospectus. Several years later, he was applying his statistical methods to the 2008 election, and the rest is history. Indeed, in this year’s election, Silver correctly called all 50 states.
Silver’s model basically works by looking at polling data from a wide variety of polls, particularly state-level polls, weighting the data based on how recent it is, sample size, and the pollster’s history of accuracy. The polling data are then used to calculate an adjusted polling average subject to trend line adjustment, house effects adjustment, and likely voter adjustment, after which they are adjusted based on various factors, such as a state’s Partisan Voting Index, and the like. Ultimately, Silver takes his aggregated poll data and uses them to run simulations that estimate the likelihood of a given outcome. That’s how he came up with his famous estimates of how likely various scenarios were. For example, the day before the election Silver was estimating that Mitt Romney only had an 8% chance of winning the Electoral College. Republicans howled. We all know the results of the election and that President Obama won reelection more easily than the conventional wisdom had been predicting. Silver also had an excellent track record predicting the Republican landslide in the 2010 midterm elections. Back then, Democrats howled.
Not surprisingly, there was considerable resistance in baseball to “moneyball” at first. The “old guard” initially didn’t like the implication that statistical modeling could judge the value of a player better than scouts, managers, and the front office. Moreover, the statistical predictions made by “moneyball”-inspired sabermetric analysis often clashed with conventional wisdom in baseball. Similarly, this year we were treated to a similar spectacle, where Nate Silver unwittingly became a political flashpoint, as his numbers, which at no point over the last couple of months favored Romney. This led to considerable resistance to the statistically-based methods of not just Nate Silver, but Sam Wang of the Princeton Election Consortium and others as well. The highest Romney’s estimated probability of winning the Electoral College according to Silver’s model ever got was 39% on October 12. That’s actually not bad, but Obama was the favorite in Silver’s model and remained the favorite up until the end. So different was Silver’s prediction from the conventional wisdom (a super-tight election that would not be called until very late on election night or might even end up with Obama winning the Electoral College but losing the popular vote) and Republican expectations (a Romney win), that on election night when the networks called Ohio for Obama not long after 11 PM, Karl Rove initially refused to accept the outcome, even though Fox News had also called it. Overall, predicting this election ended up being portrayed as a battle between data-driven nerds over ideological political pundits.
Of course, doctors are not baseball managers or ideologically-driven political pundits. Or, at least, so we would like to think. However, we are subject to the same sorts of biases as anyone else, and, unfortunately, many of us put more stock in our impressions than we do in data. Overcoming that tendency is the key challenge physicians face in embracing EBM, much less SBM. It doesn’t help that many of us are a bit too enamored of our own ability to analyze observations. As I’ve pointed out time and time again, personal clinical experience, no matter how much it might be touted by misguided physicians like, for example, Dr. Jay Gordon, who thinks that his own personal observations that lead him to believe that vaccines cause autism trump the weight of multiple epidemiological studies that do not. The same sort of dynamic occurs when it comes to “alternative” medicine (or “complementary and alternative medicine” or “integrative medicine” or whatever CAM proponents like to call it these days). At the individual level, placebo effects, regression to the mean, confirmation bias, observation bias, confusing correlation with causation, and a number of other factors can easily mislead one.
Yet, in medicine there remains resistance to EBM. Indeed, there is even an organization of physicians that explicitly rejects EBM and frequently publishes screeds against it in its major publication. This organization is known as the Association of American Physicians and Surgeons, and I’ve written about it before. For example, a former president of the AAPS, Lawrence Huntoon, MD, once dismissed EBM, represented by clinical care pathways, systems-based care, and other products of EBM as “medical herdology.” Other articles include titles such as Evidence-Based Guidelines: Not Recommended. In it, its author, Dr. Norman Latov, writes:
On its face, evidence-based medicine is just what the doctor ordered. What rational person would argue that medical decisions should not be based on evidence?
Upon closer examination, however, the term is deceptive. Evidence-based guidelines (EBGs) in fact only use evidence from controlled trials, and deny other types of evidence or clinical judgment, thereby distorting the decision process.
Indeed, when you read various critiques of EBM, they are almost inevitably different from our critiques of EBM (namely that EBM doesn’t adequately take prior probability into account). There is also a difference between CAM practitioners and apologists and doctors who think they support evidence and science in medicine. For instance, when most doctors criticize EBM, they tend to attack it because they see it as limiting their “clinical judgment” or on the basis that there are things that “EBM can’t account for.” (See the parallel here between baseball and medicine?) While it is true that there are things that EBM can’t account for, that doesn’t necessarily mean that personal experience does better. Indeed, when it’s tested rigorously, it tends not to. As for CAM practitioners, I tend to liken them to political operatives: Driven by ideology more than science, rather than simply being used to old ways of doing things, like baseball leaders in the age before “moneyball.” For instance, take a look at Andrew Weil, who most definitely does not like EBM, going so far as to write:
RCTs have dominated decision making about efficacy in health care for almost 50 years. Many researchers have explored the difficulty of subjecting IM treatment approaches to RCTs. There are some characteristics of IM interventions that make RCTs particularly difficult to carry out, and perhaps even less relevant, than for conventional allopathic medicine. As Fønnebø pointed out, the gap between published studies of integrative approaches on the one hand, and the clinical reports by practitioners on the other hand, may partially result from the fact that placebo-controlled RCTs are designed to evaluate pharmaceutical interventions.
One can almost see Karl Rove railing against Fox News analysts about calling Ohio at 11:15 PM on election night 2012 or Democratic partisans attacking Nate Silver in 2010 when he kept predicting a Republican landslide. Whatever the reasons, be they ideological, a misguided belief in their own “clinical judgment,” or just plain cussedness, there are a lot of doctors who don’t like EBM, and there are a whole lot of CAM practitioners that don’t like SBM, because CAM exploits the blind spot in EBM. But let’s get back to the analogy between moneyball and EBM:
Critics of moneyball approaches have nonetheless been quick to emphasize the way in which perspective can be distorted, not enhanced, by statistics. One might overapply concepts such as Bayes’ theorem or develop a habit of plugging data into statistical software simply to gain a patina of precision, regardless of appropriateness (tendencies that cause medical practitioners, in Alvan Feinstein’s pithy phrase, to be blinded by the “haze of Bayes”).3 Critics have also pointed to what might be termed the “uncertainty principle” of statistical analysis: general data (How well does this player hit against left-handers? How well does this therapy work in myocardial infarction?) often fail to take into account consequential distinctions; but more specific data (How well does this player hit against hard-throwing left-handers on warm Sunday afternoons in late September? How well does this therapy work in right-sided myocardial infarction in postmenopausal women?) can involve too few cases to be broadly useful. Individuals, and individual scenarios, might always be idiosyncratic on some level — a truth perhaps borne out by long-standing efforts to appropriately apply the scientific results of clinical trials to individual patients in the clinic.
Here’s where Phillips et al go wrong. As we have pointed out here on this very blog many, many times before, the problem with EBM is not that it overapplies Bayes’ theorem. Rather it’s that EBM uses frequentist statistics over Bayes’, often massively underplaying the importance of prior plausibility, estimates of which Bayesian statistics demands. Indeed, not making Baysian estimates of prior plausibility leads to EBM’s blind spot towards CAM, such that statistical noise and bias in trial design can lead to the appearance of efficacy for therapies whose rationale, to be true, would require that much of modern physics and chemistry be not just wrong but massively wrong. I’m talking, of course, about modalities like homeopathy, reiki and other “energy healing” treatments, and the like. While “moneyball” does take Bayes into account somewhat, there is one big difference between moneyball and medicine, and that’s science. There are certain things in medicine that can be dismissed as so implausible that for all practical purposes they are impossible based on physical laws and well-established science (i.e., homeopathy). The result was hilariously illustrated in a very simple way in a recent XKCD cartoon that asked if the sun just exploded.
Where Phillips et al do better is in pointing out that EBM has difficulty as the groups under study are chopped finer and finer, with the ultimate goal of developing “personalized medicine,” EBM gets harder, as does SBM. It’s also where the critics miss the boat as well. It’s just for these idiosyncratic cases where clinical judgment is still very important and EBM doesn’t banish clinical judgment. Clinical judgment instead becomes the science and art of applying medical and scientific evidence to individual patients who don’t fit the mold. They’re also correct in pointing out that another area in which moneyball can help in medicine is to increase value. Remember, the very reason moneyball was used in the first place was to help a team at a financial disadvantage compete against teams with a lot more money to burn. On the other hand, in medicine EBM and SBM are not necessarily the cheapest. It’s all tradeoffs. For instance, EBM would tell us that for women above a certain risk level for breast cancer it is better to add breast MRIs to their screening regimen, which definitely costs more. What SBM should be able to do is to produce better outcomes.
Finally, one of the biggest impediments to data-driven approaches to almost anything, be it baseball, politics, or medicine, is the perception that such approaches take away the “human touch” or “human judgment.” The problem, of course, is that human judgment is often not that reliable, given how we are so prone to cognitive quirks that lead us astray. However, as Philips et al point out, data-driven approaches need not be in conflict with recognizing the importance of contextualized judgment. After all, data-driven approaches depend on the assumptions behind the models, and we’ll never be able to take judgment out of developing the assumptions that shape the them. What the “moneyball” revolution has shown us, at least in baseball and politics, is that the opinions of experts can no longer be viewed as sacrosanct, given how often they conflict with evidence. The same is likely to be true in medicine.

NCCIH surveys physicians on their recommendation of “complementary health approaches,” with depressing results
The NCCIH recently published a study examining the percentage of US physicians who had recommended “complementary health approaches” to their patients in the last year. The percentages are far higher than they should be.
David Gorski on December 16, 2019
Link

As much as I hate to admit it, at times I feel like Sisyphus, pushing the boulder up the hill only to have it roll down the other side, only to have to push it back up the hill and roll down again, over and over and over again for all eternity. Certainly trying to keep track of pseudoscience in medicine and trying to educate why so much of so-called “complementary and alternative medicine” (CAM) is quackery (and the stuff that isn’t quackery is just medicine rebranded as CAM) and why “integrative medicine” involves integrating quackery with medicine can seem like a very Sisyphean task. I was reminded of this by a press release from the National Center for Complementary and Integrative Health (NCCIH) right before Thanksgiving that I had meant to write about before a couple of topics popped up that interested me more. Besides, the infiltration of alternative medicine quackery into medicine in the form of CAM and integrative medicine or health is ongoing, and the paper touted by NCCIH is just more evidence:
More than half of office-based physicians recommend at least one complementary health approach to their patients, according to a new analysis of data from a nationally representative survey. In addition, female physicians recommended every complementary health approach at a higher rate than male physicians except for chiropractic and osteopathic manipulation. The analysis was conducted by researchers from the National Center for Complementary and Integrative Health (NCCIH) and the National Center for Health Statistics (NCHS), and was published in The Journal of Alternative and Complementary Medicine.
Here’s the study. It is, as mentioned above, the product of researchers at NCCIH. In the introduction, the authors make an assertion that I find very startling:
Complementary and integrative health practices, interventions, and natural products include mind–body interventions such as massage therapy, acupuncture, and yoga, and natural products such as fish oil or probiotics,1 and are widely used by U.S. adults.2,3 Despite data on use of complementary health approaches (CHAs) by adults,3–6 there are no nationally representative data using a probability sample collected on the recommendations for CHAs by U.S. office-based physicians. Studies in this area have been limited to certain medical specialties (e.g., rheumatology, pediatrics, and family practice),7–14 U.S. regions,15–17 or types of CHAs (e.g., mind–body therapies and chiropractic).8,18,19 These have found recommendation rates by physicians for different CHAs were relatively high. For example, a national survey of rheumatologists found a majority were likely to recommend body work (65%), meditation (64%), and acupuncture (54%) for chronic lower back and joint pain,7 67% of pediatricians in a Seattle hospital reported recommending CHAs to patients,15 more than half of physicians in the Chesapeake area referred patients to practitioners of biofeedback and chiropractic,20 and 25% of physicians from the Hawaii Medical Service Association had/would refer patients for CHAs.16
What I found startling is that there aren’t a lot more data on the recommendation patterns for CAM over more specialties and over the whole country. After all, what has it been that NCCIH has been doing the last 25+ years? I especially wonder why, given that the figures that are published for CAM usage and recommendations are always used by advocates for an appeal to popularity, to represent CAM as being very popular and therefore worthy of study and, by implication, use. Of course, to do that, CAM advocates have to include a fair number of modalities as “CAM” that really aren’t, such as prayer, vitamins, exercise modalities (tai chi, qi gong, and yoga), and the like. This survey of doctors is no different.
So how did the investigators go about estimating what percentage of office-based physicians who recommend CAM modalities, which, for whatever reason, the authors decided to call “complementary health approaches” (CHA)? Basically, they mined the 2012 Physician Induction Interview of the National Ambulatory Medical Care Survey (NAMCS PII), a nationally representative survey of office-based physicians. The NAMCS website reveals that it’s a program of the Census Bureau. The Census Bureau describes the program thusly:
The National Ambulatory Medical Care Survey (NAMCS) is the Nation’s foremost study of ambulatory care provided at physicians’ offices and has been conducted since 1973. It focuses on visits made to non-federally employed, office-based physicians who are primarily engaged in direct patient care. Since 2006, the survey also includes an annual sample of visits to community health centers (CHCs). From 2012 through 2015, a new sampling design allowed NAMCS to make estimates not only for the nation and four Census regions, but also for up to as many as 34 of the nation’s most populous states. NAMCS provides information on patient, provider, and visit characteristics. It is sponsored by the Centers for Disease Control and Prevention’s (CDCs) National Center for Health Statistics (NCHS), and is administered by the U.S. Census Bureau.
So, the NAMCS appears to be a robust data source on the practices of office-based physicians all over the country. Here’s how it’s administered:
Each year, the National Center for Health Statistics (NCHS) scientifically selects a random group of physicians and CHCs throughout the country to participate in NAMCS. This group of physicians is selected from a list of licensed non-federal physicians in office-based practice maintained by the American Medical Association and the American Osteopathic Association. These lists are continually updated by both associations to make them as current and accurate as possible at the time of sample selection. The list of CHCs selected for participation comes from the Health Resources and Services Administration (HRSA). If you are selected, you will receive a letter in the mail from NCHS informing you of that before the Census Bureau contacts you.
Here’s where you can find some of the data produced by the NAMCS, along with some publications that resulted from its data, and here are some of the survey instruments.
Here’s how the NCCIH investigators used the NAMCS:
For the “any CHA” category, a “yes” response to the question was included, “During the past 12 months, did you recommend any of the following therapies or practices to your patients?” Therapies asked included massage therapy, herbs/other nonvitamin supplements, chiropractic/osteopathic manipulation, yoga, acupuncture, mind–body therapies (i.e., guided imagery, meditation, and progressive muscle relaxation, not including prayer), naturopathic treatment, biofeedback/hypnosis, and homeopathic treatment. Based on the literature, demographic characteristics included in the analysis were physician age, sex, race, and medical specialty (general/family practice, internal medicine, psychiatry, obstetrics/gynecology, pediatrics, and all other specialties combined), type of physician (doctor of medicine [MD] or doctor of osteopathic medicine [DO]), medical school location (United States or foreign), metropolitan statistical area (MSA)/non-MSA, and U.S. Census region (Northeast, Midwest, South, and West).7–11,15,23,24 Ethnicity was not included as there were not enough Hispanic respondents to produce reliable statistics. Because several therapies had small cell sizes prohibiting subgroup analyses, the majority of analyses were restricted to the six most prevalent individual therapies (i.e., massage therapy, herbs/nonvitamin supplements, chiropractic/osteopathic manipulation, yoga, acupuncture, and mind–body therapies), but included all nine in the “any CHA” category. To avoid small cell sizes and present statistically reliable estimates, physician’s age was collapsed into 4 categories (<45, 45–54, 55–64, and ≥65 years), race into 3 categories (white, Asian, and other), and 12 physician specialties (general surgery, orthopedic surgery, cardiovascular diseases, dermatology, urology, neurology, ophthalmology, otolaryngology, oncologists, allergists, pulmonologist, and other specialties) into 1 “other specialty” category.
There are two figures summarizing the results:
Figure 1:
Figure 2:
These results are truly depressing. Over 50% of physicians have recommended a CHA in the last year. Of course, these numbers are inflated, as they are with all such surveys, by the inclusion of a modality like massage. Massage is a perfect example of what I like to refer to as a patient support service that’s become medicalized. Massage, when properly done, feels good, and that should be enough for its use. There’s also nothing “complementary” or “integrative” about it. If massage can be used to relieve pain or loosen joints, it’s part of medicine.
I’m also not surprised that chiropractic or osteopathic manipulation was recommended by 27% of physicians. Sadly, chiropractors have been so successful at representing their specialty as a legitimate medical practice that too many doctors view it as basically interchangeable with physical therapy when, as I like to say, chiropractors are incompetent physical therapists with delusions of grandeur. While physical manipulation can indeed alleviate back pain, for instance, chiropractors claim to be able to treat systemic diseases such as asthma, allergies, diabetes, and more. They’re also very antivaccine, and their manipulation of the neck has real risks. Here’s a word of advice to my colleagues: Don’t ever refer to a chiropractor. Refer patients who might benefit from manipulation or exercise to physical therapy or occupational therapy instead of a chiropractor or an osteopath.
What really depresses me is not so much how many physicians had recommended massage or chiropractic in the 12 months before they were surveyed. Nor does it bother me as much that over one quarter of physicians had recommended vitamins, supplements, or yoga. Yes, it does bother me that so many physicians recommend vitamins and supplements without good evidence for their efficacy in anything, but it bothers me more how many physicians recommended acupuncture (22.4%), homeopathy (12.6%), and naturopathy (10.4%). I’m sorry, but if you can’t understand that homeopathy is The One Quackery To Rule Them All, with tenets underlying it that violate the laws of physics and chemistry and that would, if true, require rewriting those laws, then you should not be a physician, period. Similarly, if you don’t understand that naturopathy is a hodgepodge of pseudoscience and quackery mixed with some sensible interventions based on diet and lifestyle (although, truth be told, even the science-based interventions used by naturopaths are usually tainted with pseudoscience), you should not be a physician. As Harriet Hall has so aptly put it, what’s good about naturopathy (emphasis on healthy diet and lifestyle) is not unique to naturopathy, and what’s unique to naturopathy (quackery like homeopathy) is not good.
This survey does, however, reemphasize what I’ve been saying all along about acupuncture, as being the “gateway” quackery that lets in a lot of other stuff behind it. Many years ago, I once thought there might be something to acupuncture. The more I read, the more I studied it, though, the more I realized that it’s just a theatrical placebo and that its only effects are nonspecific placebo effects. However, unlike the case for homeopathy, where it’s pretty easy to explain why it’s nonsense, an explanation that frequently leaves medical students and fellow physicians to whom I relate it shaking their heads because they had always assumed that homeopathy was just herbal medicine, acupuncture involves actually sticking needles into the body. A lot of physicians therefore assume that there must be something to it. Sure, those who know enough about it will concede that the concepts of meridians and unblocking the flow of qi are ridiculous, but then they’ll cite claims by acupuncturists and researchers who believe in acupuncture about “releasing endorphins” or other biological mechanisms claimed for its efficacy. They don’t know about the retconned history of traditional Chinese medicine and acupuncture, its basis in astrology and bloodletting, or that the filiform needles now used in acupuncture were only introduced 80 years ago. Add to that the way advocates spin negative acupuncture studies as being positive, and I can see how physicians who don’t pay as close attention as we do to the literature might think there’s something to it, and it doesn’t help how successful acupuncturists have been in obtaining the imprimatur of the state by persuading state legislatures to license their “profession.”
The study also shows what specialties tend to recommend more CHAs. Disappointingly, at the top of the heap were general and family practice doctors, 75.5% of whom had recommended a CHA, 52.6% of whom had recommended chiropractic, and 36.1% of whom had recommended acupuncture in the last year. Internal medicine doctors weren’t much better, recommending the same CHAs at a rate of 64.8%, 39.9%, and 28.7%, respectively, with psychiatrists following close behind at a rate of 62.7%, 12.4%, and 27.7%, respectively. (I guess it’s not surprising that psychiatrists would have less call to recommend chiropractic, given the conditions they treat.) Other specialties (general surgery, orthopedic surgery, cardiovascular diseases, dermatology, urology, neurology, ophthalmology, otolaryngology, oncology, allergy, pulmonology, and other specialties) were the least likely to recommend CHAs at a rate of 29.9%, 16.3%, and 16.1%, respectively.
Table 2 in the paper and the supplemental tables, break down the rates of recommendation of CHA by a variety of factors. For example, female physicians were more likely to recommend CHAs than males (49.3% verus 63.2%, adjusted odds ratio 1.56), as were DOs (aOR 1.78), who were, unsurprisingly, also a lot more likely to recommend chiropractic or osteopathic manipulation. Oddly enough, there wasn’t an age effect on the likelihood of a physician to recommend a CHA. The aORs ranged from 0.93 to 1.23, and all the 95% confidence intervals spanned 1.0. There were differences in likelihood to recommend specific CHAs by age of physician (physicians 65 and older were less likely to recommend yoga and massage therapy, for instance), but not overall for a recommendation of any of the CHAs examined. I would have predicted that younger physicians might be more open to CAM than older physicians, and that prediction was clearly wrong. The rates of CAM recommendation are roughly equal over all the physician age cohorts examined, although the authors made it sound as though this were not the case in the discussion by citing other studies finding younger doctors to be more open to CAM. Finally, there were regional differences, with doctors in the Northeast, Midwest, and South census regions (particularly the South and Midwest; the difference was not as great for the Northeast) significantly less likely to recommend CHAs than those in the West, which intuitively does not seem surprising.
The authors conclude:
The current findings add to the literature by showing physician-based national estimates regarding sex differences in recommendations of CHA. Female physicians recommended CHAs at a higher rate than male physicians (63.2% vs. 49.3%, respectively), with this difference remaining even when controlling for additional characteristics. Furthermore, female physicians were approximately twice as likely to recommend each individual therapy, except chiropractic/osteopathic manipulation. These findings confirm and expand upon prior studies that found female physicians more likely to talk to patients about CHAs9 or recommend CHAs.8 Likely related to female physician recommendations is the fact previous studies have consistently found higher usage rates of CHAs among U.S. women than among men,5,6 and likewise found physicians who report personal usage of CHAs more likely to recommend CHAs to patients.8,15 Furthermore, previous research has found that U.S. health care workers, and, in particular, health care providers, are more likely to use CHAs than workers in other occupations.38 Given the rising rate of female physicians in the United States,39 there is reason to expect that recommendations for CHAs will increase over time.
I have seen the data that more females use CAM than males, but I just wonder if that’s a function of increased health care utilization by females in general compared to males. It is, however, rather disappointing to learn that health care workers are more likely to use CAM than the general public.
Finally:
This study analyses show that physicians often recommend CHAs, with 53.1% of office-based U.S. physicians recommending at least one to their patients in the past 12 months. Recommendation rates were higher among female physicians than among male physicians and varied by medical specialty. General/family practitioners and internists recommended CHAs at higher rates than pediatricians, and more than half of psychiatrists recommended mind–body therapies, compared with <30% of physicians from other specialties. Understanding practice patterns of U.S. physicians related to recommendations for CHAs, and their differences across medical specialties and physician sex, could inform consumers, physicians, and medical schools.
My question is: Inform them of what? How many doctors recommend quackery to their patients in addition to medicine? In reality, this study is very much like every other NCCIH study examining CAM usage: Its function is marketing, not “to inform”. Expect it to be used to argue how popular CAM is, how many doctors accept it, and thus to argue to policymakers and insurance companies that CAM should be reimbursable by government and private health insurance plans.

WHO And The Death of Evidence-Based Medicine
Steven Novella on July 1, 2026
Link

I have long been critical of the World Health Organization’s promotion of dubious treatment methods under the banner of “traditional medicine”. It now has a global strategy to promote what is calls “traditional, complementary and integrative medicine” (TCIM). In my opinion (shared by at least some others) the document is a master class in how to promote medical pseudoscience with the language of evidence-based medicine. It is very similar to RFK’s use of “gold standard science”. Superficially, it sounds fine, but in context, it is a weaponization of the weaknesses and misunderstandings of evidence-based medicine.
The goals sound reasonable, such as this paragraph:
“For the first time, the strategy presented a WHO definition of integrative medicine as “an interdisciplinary and evidence-based approach to health and well-being by using a combination of biomedical and traditional and/or complementary medical knowledge, skills and practices”. The strategy also emphasizes the need for robust scientific evidence and appropriate regulation to support the safe and effective integration of TCIM into health systems and services.”
But the document gives the game away in several critical ways, including what it says and what it doesn’t say. First, if you want to integrate rigorously evidence-based traditional medicine into mainstream medicine, I would argue this has already been done. Of course, this is a process, and I am not arguing that it is complete. Rather, science-based medicine is agnostic toward the origin or modality of any medical intervention. What matters are whether the intervention is scientifically plausible, and how robust the evidence is to indicate safety and efficacy.
Therefore, the unstated major premise of the entire document is that there are safe and effective treatments that are not being used by mainstream medicine systems for some reason. This is unlikely because how would we have rigorous evidence of safety and effectiveness if a modality were not already incorporated into mainstream medicine? Who did this research, and why? Are doctors leaving effective treatments on the table?
At best, there is an argument to be made about inefficiencies and biases in various health care delivery systems, but that is not the approach of the WHO or this document. Supporting research into treatments that are not profitable is fine. Supporting healthcare systems to provide for people who lack effective treatments is also fine.
Further, if you want to integrate science-based medicine into existing traditional health infrastructures, that can also be a way to bring more and more effective health care to many people. But the WHO wants to do the opposite – to integrate unscientific and ineffective treatments into scientific medicine.
One of my biggest concerns with this approach is captured by the frequent use of the phrase “strengthening the evidence base” for TCIM. Whenever the document refers to research, it is to shore up the evidence for TCIM. Never in the document – not once – does it refer to the use of science or evidence to determine “if” a treatment works. Never does it even entertain the notion that one goal of this project might be to eliminate treatments that do not work. It’s as if it assumes there is no such thing as a TCIM intervention that is not safe or effective.
This is consistent with the CAM world in general, and it is clear that CAM promoters have their fingerprints all over this document. First, they equate complementary, alternative and integrative language with “traditional” medicine practices. That was their entry into the WHO. Then they infuse their propaganda into the entire project – essentially hijacking support for traditional health systems to promote CAM pseudoscience and snake oil. Mission accomplished.
The other biggest indication that this is nothing but CAM promotion is their approach to evidence. While they constantly state that this project will only integrate methods that have the highest standards of evidence, they then outline lower standards of evidence (right in line with CAM propaganda).
They refer to “research approaches appropriate to the unique characteristics of TCIM.” This includes “complexity science”, which again is not necessarily a bad thing, but is easily abused. This refers to research within complex systems, where single variables are difficult to isolate. CAM proponents are good at latching onto scientific jargon and then twisting it to their own ends. In this case, they use the term “complexity science” to argue that standard research methods, like double-blind placebo-controlled trials, are not suited to studying their unique methods. You cannot control for variables, so you have to look at the entire system. This is a fancy way of justifying low-quality observational data, the kind that is rife with confounding factors like placebo effects.
Many of the methods they recommend are legitimate in themselves – when used properly. For example, they make many references to “real world” data. All of these observational techniques are helpful to show how treatments operate in practice, or to generate hypotheses. But – they are not a substitute for controlled studies, and they cannot be relied upon to determine safety and efficacy. To someone not steeped in the nuances of science-based medicine, they may sound good. But in reality they are a blaring warning sign, code for – we are not going to do the kind of science that can determine if a treatment works, only the kind that is guaranteed to show that it does work.
There is nothing in this long document to indicate that they understand any of this. They never say that if scientific evidence shows convincingly that a treatment modality is unsafe and/or ineffective they would recommend or take specific steps to eliminate it, or to protect the public from harmful or useless treatments. This is a ratchet of promotion, not a scientific process to separate what works from what doesn’t work.
For those who have been following this saga (the infiltration of medicine with snake oil) over the last half a century, there is one very telling sign – despite the evolving language, and labels, and justifications, and terminology, the modalities that are being promoted never change. It is the same list of dubious treatments – homeopathy, acupuncture, chiropractic, energy medicine, and “natural” remedies. The list never changes, because proponents are not using evidence to determine what works. They are not making genuinely new discoveries. They are just trying to find new ways to market their same-old products.
In short, this is an epic failure on the part of the WHO. They have allowed themselves to be hijacked by an agenda to promote profitable snake oil. This is ultimately entirely about supporting a fraudulent industry projected to be worth $369 billion by 2032. This will likely have a net harm to the health of the people they purport to serve.
This also represents the failure of evidence-based medicine to enforce its own standards, and to police its domain. EBM has become a marketing term. It means whatever people want it to mean, which is often – I can cherry-pick evidence to show that my treatment works; therefore it’s evidence-based, even if the treatment itself makes zero scientific sense.

